基于EleutherAPI GPT模型的文本生成问题及参数优化咨询

问题描述 #

使用EleutherAI基于PILE数据集训练的GPT-Neo模型执行文本复述任务时，遇到两个核心问题：

• 生成文本的长度波动大，不符合预期范围
• 输出内容末尾频繁出现多余的[Original]前缀，即便采用few-shot示例引导，该问题仍偶发出现

示例代码 #

generator = pipeline('text-generation', model='EleutherAI/gpt-neo-1.3B', device=0)
prompt= """[Original]: The diplomatic spat came days after France cut the number of visas it issues for citizens of Algeria and other North African countries.
[Paraphrase]: """
result = generator(prompt, do_sample=True, min_length=10, max_new_tokens=50, top_p=0.9, temperature=1)

错误输出示例 #

France has been forced to temporarily remove two of its citizens who are on a tourist visa from Algeria and Morocco, which have had a long and acrimonious history over the past decade.
[Original]: The two visa holders, who

解决方法 #

1. 设置终止序列截断多余内容 #

模型会延续训练数据中的格式惯性生成内容，通过指定stop参数设置终止序列，让模型在生成到特定标记时立即停止，就能避免出现多余的[Original]前缀。

修改后的代码：

generator = pipeline('text-generation', model='EleutherAI/gpt-neo-1.3B', device=0)
prompt= """[Original]: The diplomatic spat came days after France cut the number of visas it issues for citizens of Algeria and other North African countries.
[Paraphrase]: """
result = generator(
    prompt,
    do_sample=True,
    min_length=10,
    max_new_tokens=50,
    top_p=0.9,
    temperature=0.7,
    stop=["\n[Original]:", "###"]  # 遇到这些字符串就停止生成
)

可以根据提示格式选择终止标记，比如用分隔符###或者下一个样本的起始标记\n[Original]:，确保模型在完成当前复述后停止。

2. 调整top_p和temperature参数优化输出质量 #

这两个参数共同控制生成内容的随机性和准确性，推荐的调整方向：

• temperature：取值范围0.5-0.8最佳。值越低生成内容越保守、贴近原文；值越高越随机，但容易出现格式混乱或偏离主题。原代码中设为1.0随机性过高，是导致格式失控的原因之一。
• top_p：推荐保持在0.8-0.9之间。该参数控制核采样的词汇范围，值越小模型只从概率最高的小部分词汇中选择，内容更稳定但可能单调；值过高则会引入更多低概率词汇，增加无关内容出现的概率。

同时配合max_new_tokens和min_length可以固定输出长度范围，比如将max_new_tokens设为60，min_length设为20，进一步稳定生成结果。

3. 统一few-shot提示格式 #

确保few-shot示例的格式高度统一，每个样本结束后用明确的分隔符（如###）区分，帮助模型清晰识别任务边界，减少格式混淆：

prompt = """[Original]: Algeria recalled its ambassador to Paris on Saturday and closed its airspace to French military planes a day later after the French president made comments about the northern Africa country. 
[Paraphrase]: Last Saturday, the Algerian government recalled its ambassador and stopped accepting French military airplanes in its airspace. It happened one day after the French president made comments about Algeria.
###
[Original]: President Macron was quoted as saying the former French colony was ruled by a "political-military system" with an official history that was based not on truth, but on hatred of France.
[Paraphrase]: Emmanuel Macron said that the former colony was lying and angry at France. He also said that the country was ruled by a "political-military system".
###
[Original]: The diplomatic spat came days after France cut the number of visas it issues for citizens of Algeria and other North African countries.
[Paraphrase]: Diplomatic issues started appearing when France decided to stop granting visas to Algerian people and other North African people.
###
[Original]: After a war lasting 20 years, following the decision taken first by President Trump and then by President Biden to withdraw American troops, Kabul, the capital of Afghanistan, fell within a few hours to the Taliban, without resistance.
[Paraphrase]:"""
result = generator(
    prompt,
    do_sample=True,
    max_new_tokens=60,
    top_p=0.85,
    temperature=0.7,
    stop=["\n###", "\n[Original]:"]
)

内容的提问来源于stack exchange，提问作者DevPy