如何将GPT2用作问答系统？上下文该如何设置？

问题描述 #

尝试用PyTorch基于GPT2实现问答系统，使用Hugging Face示例代码后发现，必须指定context才能运行问答pipeline，返回的答案就是提供的上下文，无法利用GPT2预训练的知识直接生成问题答案，想知道是否有无需提前给上下文的实现方法，或者是否应该用维基百科全量内容作为上下文。

当前使用代码 #

from transformers import AutoTokenizer, GPT2ForQuestionAnswering
import torch
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2-large")

model = GPT2ForQuestionAnswering.from_pretrained("openai-community/gpt2-large")

question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"

inputs = tokenizer(question, text, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

answer_start_index = outputs.start_logits.argmax()
answer_end_index = outputs.end_logits.argmax()

predict_answer_tokens = inputs.input_ids[0, answer_start_index : answer_end_index + 1]

# target is "nice puppet"
target_start_index = torch.tensor([14])
target_end_index = torch.tensor([15])

outputs = model(**inputs, start_positions=target_start_index, end_positions=target_end_index)
loss = outputs.loss

question_answerer = pipeline("question-answering", model=model, tokenizer=tokenizer)
question_answerer = question_answerer(question=question, context = text)
print(question_answerer)

运行输出 #

Some weights of GPT2ForQuestionAnswering were not initialized from the model checkpoint at openai-community/gpt2-large and are newly initialized: ['qa_outputs.bias', 'qa_outputs.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. 
{'score': 0.05718426778912544, 'start': 0, 'end': 28, 'answer': 'Jim Henson was a nice puppet'}

原因分析 #

当前使用的GPT2ForQuestionAnswering是抽取式问答模型，核心逻辑是从给定的context中抽取答案片段，因此必须传入context才能运行。而且直接用预训练GPT2初始化该模型时，新增的问答头部层（qa_outputs）是随机初始化的，未经过下游问答任务微调，导致模型无法正确定位答案位置，直接返回了整个上下文。

解决方案 #

1. 改用生成式问答（无需提前给上下文） #

GPT2本质是自回归语言模型，预训练时已学习了海量文本知识，无需额外传入上下文，只需构造合适的prompt让模型直接生成答案。具体实现方式如下：

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

# 加载tokenizer和生成式模型
tokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2-large")
# GPT2默认没有pad_token，用eos_token替代
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained("openai-community/gpt2-large")

# 构造问答prompt，引导模型生成答案
question = "Who was Jim Henson?"
prompt = f"Question: {question}\nAnswer:"

# 使用文本生成pipeline
generator = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=100,  # 控制生成答案的长度
    temperature=0.7,     # 控制生成的随机性，值越低越保守
    top_p=0.9            # 核采样，提升生成质量
)

# 生成并打印结果
output = generator(prompt)
print(output[0]['generated_text'])

运行后模型会基于预训练知识生成类似如下的答案：

Question: Who was Jim Henson?
Answer: Jim Henson was an American puppeteer, filmmaker, and television producer. He is best known as the creator of the Muppets, including beloved characters like Kermit the Frog and Miss Piggy. Henson worked on numerous television shows and films, such as "Sesame Street" and "The Muppet Movie," leaving a lasting impact on children's entertainment.

2. 关于维基百科全量内容的使用建议 #

不建议直接将维基百科全量内容作为context，因为GPT2的上下文窗口最大仅为1024 tokens，无法容纳大规模文本。如果需要结合外部知识库提升答案准确性，建议采用**检索增强生成（RAG）**方案：

• 先根据问题检索维基百科中与问题相关的片段
• 将这些片段作为补充信息，和问题一起构造prompt输入模型生成答案

这种方式既利用了外部知识库的精准信息，又符合模型的上下文窗口限制，能有效提升问答质量。

内容的提问来源于stack exchange，提问作者meinchef