M2 macOS上微调Llama2遇AttributeError: can't set attribute错误求助

问题场景 #

在M2芯片的macOS系统上微调llama2-7b-chat-hf模型，运行以下代码时触发错误：

tokenizer = AutoTokenizer.from_pretrained(base_model, use_fast=True)
tokenizer.pad_token = tokenizer.unk_token
tokenizer.padding_side = "right"

错误栈 #

Traceback (most recent call last):
  File "/Users/salmakhaled/Pobot/venv/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3550, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/var/folders/67/wk5jj9q91n3g7llmbclsf6wm0000gp/T/ipykernel_24144/1335867970.py", line 2, in <module>
    tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
  File "/Users/salmakhaled/Pobot/venv/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 787, in from_pretrained
  File "/Users/salmakhaled/Pobot/venv/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 2028, in from_pretrained
    if isinstance(value, dict):
  File "/Users/salmakhaled/Pobot/venv/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 2260, in _from_pretrained
    save_directory, (filename_prefix + "-" if filename_prefix else "") + ADDED_TOKENS_FILE
  File "/Users/salmakhaled/Pobot/venv/lib/python3.9/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 129, in __init__
    self.can_save_slow_tokenizer = False if not self.vocab_file else True
AttributeError: can't set attribute

环境配置 #

• Python 3.9.6
• transformers 4.36.1
• accelerate 0.25.0
• torch 2.3.0.dev20240118

解决方法 #

方法1：降级transformers版本 #

transformers 4.36.x版本中，LlamaFastTokenizer的can_save_slow_tokenizer被改为只读的@property属性，但初始化逻辑仍尝试直接赋值，导致冲突。降级到4.35.2版本可解决该问题：

pip install transformers==4.35.2

方法2：修改transformers源码 #

如果不想降级，可直接修改对应文件的初始化代码：

1. 找到虚拟环境中tokenization_llama_fast.py的路径：/Users/salmakhaled/Pobot/venv/lib/python3.9/site-packages/transformers/models/llama/tokenization_llama_fast.py
2. 定位到第129行：self.can_save_slow_tokenizer = False if not self.vocab_file else True
3. 注释或删除该行代码（该属性现在由@property自动管理，无需手动赋值）

方法3：禁用Fast Tokenizer #

暂时放弃使用Fast Tokenizer，将加载代码中的use_fast=True改为use_fast=False：

tokenizer = AutoTokenizer.from_pretrained(base_model, use_fast=False)
tokenizer.pad_token = tokenizer.unk_token
tokenizer.padding_side = "right"

此方法会降低tokenizer速度，但能快速绕过错误。

内容的提问来源于stack exchange，提问作者Salma Khaled