使用Hugging Face微调文本生成模型时遇ValueError：模型未返回loss

我使用PyTorch结合transformers库微调Hugging Face模型，环境配置为torch版本1.13.0+cu117、Python 3.7.8、CUDA 11.8。复制代码后出现报错：

ValueError: The model did not return a loss from the inputs, only the following keys: logits,past_key_values. For reference, the inputs it received are input_ids,token_type_ids,attention_mask.

我做的是文本生成任务，使用AutoModelForCausalLM而非GPT2专用生成模型。报错回溯信息如下：

ValueError                                Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_8240\3982389964.py in <module>
     47 )
     48 
---> 49 trainer.train()

c:\users\fchen\appdata\local\programs\python\python37\lib\site-packages\transformers\trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1503             resume_from_checkpoint=resume_from_checkpoint,
   1504             trial=trial,
-> 1505             ignore_keys_for_eval=ignore_keys_for_eval,
   1506         )
   1507 

c:\users\fchen\appdata\local\programs\python\python37\lib\site-packages\transformers\trainer.py in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1747                         tr_loss_step = self.training_step(model, inputs)
   1748                 else:
-> 1749                     tr_loss_step = self.training_step(model, inputs)
   1750 
   1751                 if (

c:\users\fchen\appdata\local\programs\python\python37\lib\site-packages\transformers\trainer.py in training_step(self, model, inputs)
   2506 
   2507         with self.compute_loss_context_manager():
-> 2508             loss = self.compute_loss(model, inputs)
   2509 
   2510         if self.args.n_gpu > 1:

c:\users\fchen\appdata\local\programs\python\python37\lib\site-packages\transformers\trainer.py in compute_loss(self, model, inputs, return_outputs)
   2552             if isinstance(outputs, dict) and "loss" not in outputs:
   2553                 raise ValueError(
-> 2554                     "The model did not return a loss from the inputs, only the following keys: "
   2555                     f"{','.join(outputs.keys())}. For reference, the inputs it received are {','.join(inputs.keys())}."
   2556                 )

ValueError: The model did not return a loss from the inputs, only the following keys: logits,past_key_values. For reference, the inputs it received are input_ids,token_type_ids,attention_mask.

完整代码如下（train.csv和test.csv每行存储一句自然语言）：

from transformers import TrainingArguments, Trainer
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import TrainingArguments
from datasets import load_dataset
import evaluate

MAX_LEN=100

def tokenize_function(examples):
    return tokenizer(examples["sentences"], padding='max_length', truncation=True,max_length=MAX_LEN)

pretrained = "./models/gpt2-chinese-cluecorpussmall/"
tokenizer = AutoTokenizer.from_pretrained(pretrained)
model = AutoModelForCausalLM.from_pretrained(pretrained)

data_files = {"train": "train.csv", "test": "test.csv"}
dataset = load_dataset("csv", data_files=data_files)

tokenized_datasets = dataset.map(tokenize_function, batched=True)

train_dataset = tokenized_datasets["train"].shuffle(seed=42)
eval_dataset = tokenized_datasets["test"].shuffle(seed=42)

training_args = TrainingArguments(
    output_dir='./test_trainer',
    num_train_epochs=1,
    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    learning_rate= 5e-05,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir='./logs',
    load_best_model_at_end=True,
    logging_steps=400,         
    save_steps=400,            
    evaluation_strategy="steps",
    report_to=None
)

training_args = TrainingArguments(output_dir="test_trainer", evaluation_strategy="epoch")

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset =eval_dataset 
)

trainer.train()

核心原因 #

1. 缺少labels输入字段：因果语言模型（CausalLM）计算自回归损失需要将input_ids作为标签传入，当前数据集未生成labels字段，模型无法自动计算损失。
2. TrainingArguments被重复覆盖：先定义的完整训练配置被后续极简配置覆盖，导致之前的参数全部失效。

修复步骤 #

1. 修改分词函数，添加并优化labels字段

因果语言模型的训练标签就是输入本身（自回归预测下一个token），同时将padding位置的标签设为-100（Hugging Face会自动忽略这些位置的损失计算）：

def tokenize_function(examples):
    tokenized = tokenizer(examples["sentences"], padding='max_length', truncation=True, max_length=MAX_LEN)
    # 生成labels并处理padding
    tokenized["labels"] = [
        [-100 if token == tokenizer.pad_token_id else label for token, label in zip(input_id, label_id)]
        for input_id, label_id in zip(tokenized["input_ids"], tokenized["input_ids"])
    ]
    return tokenized

2. 保留完整的TrainingArguments配置

删除重复的配置赋值语句，保留第一个完整的训练参数设置：

# 保留完整配置
training_args = TrainingArguments(
    output_dir='./test_trainer',
    num_train_epochs=1,
    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    learning_rate= 5e-05,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir='./logs',
    load_best_model_at_end=True,
    logging_steps=400,         
    save_steps=400,            
    evaluation_strategy="steps",
    report_to=None
)
# 删除下面这行重复赋值
# training_args = TrainingArguments(output_dir="test_trainer", evaluation_strategy="epoch")

3. 补充GPT2中文模型的pad_token（可选但必要）

部分GPT2中文模型默认没有pad_token，需要手动设置：

tokenizer = AutoTokenizer.from_pretrained(pretrained)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

修复后完整代码 #

from transformers import TrainingArguments, Trainer
from transformers import AutoTokenizer, AutoModelForCausalLM
from datasets import load_dataset
import evaluate

MAX_LEN=100

def tokenize_function(examples):
    tokenized = tokenizer(examples["sentences"], padding='max_length', truncation=True, max_length=MAX_LEN)
    # 生成labels并处理padding
    tokenized["labels"] = [
        [-100 if token == tokenizer.pad_token_id else label for token, label in zip(input_id, label_id)]
        for input_id, label_id in zip(tokenized["input_ids"], tokenized["input_ids"])
    ]
    return tokenized

pretrained = "./models/gpt2-chinese-cluecorpussmall/"
tokenizer = AutoTokenizer.from_pretrained(pretrained)
# 手动设置pad_token
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(pretrained)

data_files = {"train": "train.csv", "test": "test.csv"}
dataset = load_dataset("csv", data_files=data_files)

tokenized_datasets = dataset.map(tokenize_function, batched=True)

train_dataset = tokenized_datasets["train"].shuffle(seed=42)
eval_dataset = tokenized_datasets["test"].shuffle(seed=42)

training_args = TrainingArguments(
    output_dir='./test_trainer',
    num_train_epochs=1,
    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    learning_rate= 5e-05,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir='./logs',
    load_best_model_at_end=True,
    logging_steps=400,         
    save_steps=400,            
    evaluation_strategy="steps",
    report_to=None
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset 
)

trainer.train()

内容的提问来源于stack exchange，提问作者fchen92