GPT2微调问答模型生成异常答案，寻求技术排查帮助

问题概述 #

使用给定的faq.txt微调GPT2模型实现问答功能，但推理阶段生成结果为!!! !!!，无法输出正确答案。

核心问题分析 #

1. 数据格式不匹配：训练数据中问题带单引号（如Q: 'Where is Paris?'），但推理输入格式为Q: Where is Paris?，模型未学习到一致的问答模式，导致无法正确解码。
2. 缺少Pad Token配置：GPT2 tokenizer默认无pad token，原代码中CrossEntropyLoss指定ignore_index=tokenizer.pad_token_id，但该值为None，导致loss计算错误，训练无效。
3. 训练数据构造错误：原代码将问题和答案拆分为独立的输入和目标，不符合GPT2自回归模型的训练逻辑，模型无法学习从问题生成答案的关联关系。
4. 训练数据量不足+epoch数过少：仅2条训练数据，仅训练3个epoch，模型无法充分学习到问答映射规则。

解决方案 #

1. 统一数据格式 #

修改faq.txt，移除问题和答案中的单引号，保持与推理输入格式一致：

Q: Where is Paris?
A: Paris is in France.
Q: Where is Athens?
A: Greece

2. 配置Token Pad Token #

GPT2默认无pad token，将eos_token设为pad token，避免batch训练时的长度不匹配问题：

tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
tokenizer.pad_token = tokenizer.eos_token  # 设置pad token为eos token

3. 重构训练数据构造逻辑 #

将问题与答案拼接为完整的prompt序列，让模型学习从问题前缀生成答案的自回归逻辑，同时对batch内的样本做padding：

class FAQDataset(Dataset):
    def __init__(self, data_file, tokenizer, max_len=100):
        self.tokenizer = tokenizer
        self.max_len = max_len
        self.data = []

        with open(data_file, 'r') as file:
            lines = file.readlines()
            for i in range(0, len(lines)-1, 2):
                question = lines[i].strip()
                answer = lines[i+1].strip()
                # 拼接为完整的问答序列
                full_text = f"{question}\n{answer}"
                self.data.append(full_text)

    def __len__(self):
        return len(self.data)

    def __getitem__(self, index):
        text = self.data[index]
        # 编码并padding到max_len
        encoding = self.tokenizer.encode_plus(
            text,
            add_special_tokens=True,
            max_length=self.max_len,
            padding='max_length',
            truncation=True,
            return_tensors='pt'
        )
        input_ids = encoding['input_ids'].squeeze()
        attention_mask = encoding['attention_mask'].squeeze()
        # 标签与输入一致，GPT2会自动处理shifted label
        labels = input_ids.clone()
        # 忽略pad token的loss
        labels[input_ids == self.tokenizer.pad_token_id] = -100
        return {
            'input_ids': input_ids,
            'attention_mask': attention_mask,
            'labels': labels
        }

4. 调整训练参数 #

增加epoch数至10，同时适配小数据量调整batch size：

# 训练参数调整
batch_size = 2  # 数据量小，batch设为2
num_epochs = 10
learning_rate = 1e-4  # 适当提高学习率

# 数据加载器
data_loader = DataLoader(dataset, batch_size=batch_size, shuffle=True)

5. 优化推理生成参数 #

调整生成参数避免重复或乱码，同时确保输入格式与训练一致：

# 推理代码修改
user_question = "Where is Paris?"
# 输入格式与训练时一致
input_text = f"Q: {user_question}\nA:"
input_ids = tokenizer.encode(input_text, return_tensors='pt')

# 生成时设置参数
output = model.generate(
    input_ids,
    max_length=100,
    num_return_sequences=1,
    temperature=0.7,  # 控制生成随机性
    top_p=0.9,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
    do_sample=True
)
generated_answer = tokenizer.decode(output[:, input_ids.shape[-1]:][0], skip_special_tokens=True).strip()
print(generated_answer)

修改后的完整代码 #

微调代码 #

import torch
from torch.utils.data import Dataset, DataLoader
from transformers import GPT2LMHeadModel, GPT2Tokenizer

class FAQDataset(Dataset):
    def __init__(self, data_file, tokenizer, max_len=100):
        self.tokenizer = tokenizer
        self.max_len = max_len
        self.data = []

        with open(data_file, 'r') as file:
            lines = file.readlines()
            for i in range(0, len(lines)-1, 2):
                question = lines[i].strip()
                answer = lines[i+1].strip()
                full_text = f"{question}\n{answer}"
                self.data.append(full_text)

    def __len__(self):
        return len(self.data)

    def __getitem__(self, index):
        text = self.data[index]
        encoding = self.tokenizer.encode_plus(
            text,
            add_special_tokens=True,
            max_length=self.max_len,
            padding='max_length',
            truncation=True,
            return_tensors='pt'
        )
        input_ids = encoding['input_ids'].squeeze()
        attention_mask = encoding['attention_mask'].squeeze()
        labels = input_ids.clone()
        labels[input_ids == self.tokenizer.pad_token_id] = -100
        return {
            'input_ids': input_ids,
            'attention_mask': attention_mask,
            'labels': labels
        }

# 加载tokenizer和模型，设置pad token
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
tokenizer.pad_token = tokenizer.eos_token
model = GPT2LMHeadModel.from_pretrained('gpt2')

# 加载数据集
dataset = FAQDataset('faq.txt', tokenizer)

# 训练参数
batch_size = 2
num_epochs = 10
learning_rate = 1e-4

data_loader = DataLoader(dataset, batch_size=batch_size, shuffle=True)

model.train()
optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)

# 训练循环
for epoch in range(num_epochs):
    total_loss = 0
    for batch in data_loader:
        optimizer.zero_grad()
        outputs = model(
            input_ids=batch['input_ids'],
            attention_mask=batch['attention_mask'],
            labels=batch['labels']
        )
        loss = outputs.loss
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    avg_loss = total_loss / len(data_loader)
    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {avg_loss:.4f}")

# 保存模型
model.save_pretrained('fine-tuned-gpt2')
tokenizer.save_pretrained('fine-tuned-gpt2')

推理代码 #

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = GPT2LMHeadModel.from_pretrained('fine-tuned-gpt2')
tokenizer = GPT2Tokenizer.from_pretrained('fine-tuned-gpt2')

model.eval()

user_question = "Where is Paris?"
input_text = f"Q: {user_question}\nA:"
input_ids = tokenizer.encode(input_text, return_tensors='pt')

with torch.no_grad():
    output = model.generate(
        input_ids,
        max_length=100,
        num_return_sequences=1,
        temperature=0.7,
        top_p=0.9,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
        do_sample=True
    )

generated_answer = tokenizer.decode(output[:, input_ids.shape[-1]:][0], skip_special_tokens=True).strip()
print(generated_answer)  # 预期输出：Paris is in France.

验证结果 #

修改后重新训练并推理，针对问题Where is Paris?，模型应生成Paris is in France.；针对Where is Athens?，应生成Greece。

内容的提问来源于stack exchange，提问作者tagg