使用Hugging Face Transformers调用GPT2生成文本遇属性错误求助

问题描述 #

使用GPT2Tokenizer和AutoModelForCausalLM进行文本生成时，先后尝试了transformers==4.10.0、transformers==4.30.2版本，甚至通过--upgrade git+https://github.com/huggingface/transformers.git升级，仍报错：

AttributeError: 'GPT2LMHeadModel' object has no attribute 'compute_transition_scores'

运行代码 #

from transformers import GPT2Tokenizer, AutoModelForCausalLM
import numpy as np
import pandas as pd


x = "sample Text" #df_toxic['text'].iloc[0]

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer.pad_token_id = tokenizer.eos_token_id
inputs = tokenizer(x, return_tensors="pt")

# Example 1: Print the scores for each token generated with Greedy Search
outputs = model.generate(**inputs, max_new_tokens=5, return_dict_in_generate=True, output_scores=True)
transition_scores = model.compute_transition_scores(
    outputs.sequences, outputs.scores, normalize_logits=True
)
# input_length is the length of the input prompt for decoder-only models, like the GPT family, and 1 for
# encoder-decoder models, like BART or T5.
input_length = 1 if model.config.is_encoder_decoder else inputs.input_ids.shape[1]
generated_tokens = outputs.sequences[:, input_length:]
for tok, score in zip(generated_tokens[0], transition_scores[0]):
    # | token | token string | logits | probability
    print(f"| {tok:5d} | {tokenizer.decode(tok):8s} | {score.numpy():.3f} | {np.exp(score.numpy()):.2%}")

错误信息 #

Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [21], line 3
      1 # Example 1: Print the scores for each token generated with Greedy Search
      2 outputs = model.generate(**inputs, max_new_tokens=5, return_dict_in_generate=True, output_scores=True)
----> 3 transition_scores = model.compute_transition_scores(
      4     outputs.sequences, outputs.scores, normalize_logits=True
      5 )
      6 # # input_length is the length of the input prompt for decoder-only models, like the GPT family, and 1 for
      7 # # encoder-decoder models, like BART or T5.
      8 # input_length = 1 if model.config.is_encoder_decoder else inputs.input_ids.shape[1]
   (...)
     11 #     # | token | token string | logits | probability
     12 #     print(f"| {tok:5d} | {tokenizer.decode(tok):8s} | {score.numpy():.3f} | {np.exp(score.numpy()):.2%}")

File /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1207, in Module.__getattr__(self, name)
   1205     if name in modules:
   1206         return modules[name]
-> 1207 raise AttributeError("'{}' object has no attribute '{}'".format(
   1208     type(self).__name__, name))

AttributeError: 'GPT2LMHeadModel' object has no attribute 'compute_transition_scores'

解决方案 #

1. 升级到支持该方法的transformers版本 #

compute_transition_scores方法是在transformers 4.31.0版本才正式添加到GPT2LMHeadModel中的，你之前尝试的4.30.2及更早版本都不支持这个方法。执行以下命令升级：

pip install --upgrade transformers>=4.31.0

升级后可以通过以下代码确认版本：

import transformers
print(transformers.__version__)

确保输出为4.31.0或更高版本。

2. 手动实现过渡分数计算（无需依赖新版本） #

如果暂时无法升级版本，可以手动基于outputs.scores计算过渡分数，替换后的完整代码如下：

from transformers import GPT2Tokenizer, AutoModelForCausalLM
import numpy as np
import pandas as pd
import torch.nn.functional as F


x = "sample Text" #df_toxic['text'].iloc[0]

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer.pad_token_id = tokenizer.eos_token_id
inputs = tokenizer(x, return_tensors="pt")

# 生成文本并获取分数
outputs = model.generate(**inputs, max_new_tokens=5, return_dict_in_generate=True, output_scores=True)

input_length = 1 if model.config.is_encoder_decoder else inputs.input_ids.shape[1]
generated_tokens = outputs.sequences[:, input_length:]

# 手动计算过渡分数
transition_scores = []
for idx, score in enumerate(outputs.scores):
    # 对logits做log_softmax归一化，取对应生成token的分数
    log_probs = F.log_softmax(score, dim=-1)
    token_score = log_probs[0, generated_tokens[0][idx]]
    transition_scores.append(token_score)

# 转换为和原方法一致的张量格式
transition_scores = torch.tensor(transition_scores).unsqueeze(0)

# 打印结果
for tok, score in zip(generated_tokens[0], transition_scores[0]):
    print(f"| {tok:5d} | {tokenizer.decode(tok):8s} | {score.numpy():.3f} | {np.exp(score.numpy()):.2%}")

内容的提问来源于stack exchange，提问作者user11849691