基于Hugging Face的GPT-2少样本提示工程：文本改写实现问题

问题描述 #

我想用Hugging Face实现文本改写功能：句子由前缀、消息、后缀三部分组成，用@分隔，要求保留前缀和后缀，仅改写中间的消息部分。示例如下：

• 输入："Your bravery @ is wasted on @ those people." → 输出："Your bravery @ is useless for @ those people."
• 输入："This guy @ is meticulous in both planning @ and execution." → 输出："This guy @ is diligent when it comes to preparation @ and execution."

我写了少样本提示词，但代码无法正常工作，原代码如下：

torch.manual_seed(0)
model = "gpt2"

tokenizer = AutoTokenizer.from_pretrained(model)
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = """Sentence: It @ sucks that you got caught, @ but it's not my fault.
Rewritten: It @ is too bad you were exposed, @ but it's not my fault.
Sentence: Your bravery @ is wasted on @ those people.
Rewritten: Your bravery @ is useless for @ those people.
Sentence: This guy @ is meticulous in both planning @ and execution.
Rewritten: This guy @ is diligent when it comes to preparation @ and execution.
Rewrite the text between the 2 @ symbols in the following sentence.
Sentence: It @ sucks that you got caught, @ but it's not my fault.
Rewritten: 
"""

sequences = pipe(
    prompt,
    max_new_tokens=10,
)

for seq in sequences:
    print(f"Result: {seq['generated_text']}")

问题分析与修复方案 #

1. 模型适配问题 #

GPT-2是基础续写模型，对指令理解和少样本任务的支持较弱，建议换成更适合指令任务的模型，比如轻量版的distilgpt2，或者有API权限的话用gpt-3.5-turbo-instruct，开源模型可选Llama-2-7b-chat-hf这类对话模型。

2. 提示词优化 #

原提示词指令模糊，还重复第一个示例作为测试输入，容易让模型混淆。可以简化提示，明确强化“保留前后@外内容，只改中间”的规则：

你需要完成文本改写任务：句子由前缀、待改写内容、后缀组成，三者用@分隔。请完全保留前缀和后缀，仅改写中间的待改写内容，改写后语义要和原内容一致。

示例1：
输入：Your bravery @ is wasted on @ those people.
输出：Your bravery @ is useless for @ those people.

示例2：
输入：This guy @ is meticulous in both planning @ and execution.
输出：This guy @ is diligent when it comes to preparation @ and execution.

现在处理以下句子：
输入：It @ sucks that you got caught, @ but it's not my fault.
输出：

3. 生成参数调整 #

原代码max_new_tokens=10太小，不足以生成完整改写内容；需要添加参数控制生成随机性，同时设置终止符避免无意义续写。

修复后的完整代码 #

import torch
from transformers import AutoTokenizer, pipeline

torch.manual_seed(0)
# 换成更适配指令任务的模型
model = "distilgpt2"

tokenizer = AutoTokenizer.from_pretrained(model)
# 补充pad_token避免警告
tokenizer.pad_token = tokenizer.eos_token

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# 优化后的提示词
prompt = """你需要完成文本改写任务：句子由前缀、待改写内容、后缀组成，三者用@分隔。请完全保留前缀和后缀，仅改写中间的待改写内容，改写后语义要和原内容一致。

示例1：
输入：Your bravery @ is wasted on @ those people.
输出：Your bravery @ is useless for @ those people.

示例2：
输入：This guy @ is meticulous in both planning @ and execution.
输出：This guy @ is diligent when it comes to preparation @ and execution.

现在处理以下句子：
输入：It @ sucks that you got caught, @ but it's not my fault.
输出：
"""

sequences = pipe(
    prompt,
    max_new_tokens=50,  # 增大token数确保生成完整内容
    temperature=0.7,    # 控制生成随机性
    top_p=0.9,
    do_sample=True,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id
)

for seq in sequences:
    # 提取最终生成的改写结果
    result = seq['generated_text'].split("输出：")[-1].strip()
    print(f"改写结果：{result}")

额外说明 #

如果使用闭源模型如gpt-3.5-turbo-instruct，只需替换model参数，提示词可以更简洁，模型的指令理解能力会更强，改写效果也更稳定。

内容的提问来源于stack exchange，提问作者justinxhan