本地运行srimanth-d/GOT_CPU模型遇attention_mask等报错求解决

错误信息 #

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id:None for open-end generation. The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. The seen_tokens attribute is deprecated and will be removed in v4.41. Use the cache_position model input instead.

原运行代码 #

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('srimanth-d/GOT_CPU', trust_remote_code=True)
model = AutoModel.from_pretrained('srimanth-d/GOT_CPU', trust_remote_code=True, low_cpu_mem_usage=True, use_safetensors=True, pad_token_id=tokenizer.eos_token_id)
model = model.eval()

image_file = 'images.png'

res = model.chat(tokenizer, image_file, ocr_type='ocr')

print(res)

解决方案 #

1. 为Tokenizer明确设置Pad Token #

原tokenizer未初始化pad_token，导致模型无法正确识别填充标记，需手动指定：

tokenizer = AutoTokenizer.from_pretrained('srimanth-d/GOT_CPU', trust_remote_code=True)
# 将eos_token设为pad_token
tokenizer.pad_token = tokenizer.eos_token

2. 同步模型配置的Pad Token ID #

加载模型时，除了传入pad_token_id，还需显式更新模型配置中的对应参数：

model = AutoModel.from_pretrained(
    'srimanth-d/GOT_CPU', 
    trust_remote_code=True, 
    low_cpu_mem_usage=True, 
    use_safetensors=True,
    pad_token_id=tokenizer.pad_token_id
)
model.config.pad_token_id = tokenizer.pad_token_id

3. 手动生成并传递Attention Mask #

由于pad token与eos token相同，模型无法自动推断attention mask。若chat方法支持传入该参数，可在调用时手动生成并传递；若不支持，可改用generate方法直接处理输入：

# 示例：改用generate方法（需根据模型输入格式调整）
# 1. 处理图像输入（参考模型chat方法的内部逻辑）
# 2. 构建文本输入并生成attention_mask
text_input = ""  # 根据模型需求设置文本提示
inputs = tokenizer(text_input, return_tensors="pt", padding=True)
inputs['attention_mask'] = tokenizer.get_attention_mask(inputs['input_ids'])
# 3. 加入图像输入（需符合模型要求的格式）
inputs['image'] = ...  # 加载并预处理图像
# 4. 调用generate
outputs = model.generate(**inputs)
res = tokenizer.decode(outputs[0], skip_special_tokens=True)

4. 处理seen_tokens废弃警告 #

该警告源于transformers版本迭代，可升级transformers至最新版本；若无法修改模型远程代码，可暂时忽略该警告（不影响核心功能运行）。

完整修改后代码 #

from transformers import AutoModel, AutoTokenizer

# 初始化Tokenizer并设置pad_token
tokenizer = AutoTokenizer.from_pretrained('srimanth-d/GOT_CPU', trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token

# 加载模型并同步pad_token_id配置
model = AutoModel.from_pretrained(
    'srimanth-d/GOT_CPU', 
    trust_remote_code=True, 
    low_cpu_mem_usage=True, 
    use_safetensors=True,
    pad_token_id=tokenizer.pad_token_id
)
model.config.pad_token_id = tokenizer.pad_token_id
model = model.eval()

image_file = 'images.png'

# 优先尝试chat方法，若仍有问题则改用上述generate逻辑
res = model.chat(tokenizer, image_file, ocr_type='ocr')

print(res)

内容的提问来源于stack exchange，提问作者Yadnyesh-Dashpute