OpenAI GPT-3 API返回无关响应的技术排查求助

调用OpenAI GPT-3文本补全API时，传入问题后模型未返回对应回答，反而生成更多同类问题。尝试调整所有temperature参数值，测试davinci-codex、davinci、curie、babbage等全部模型，结果始终与ChatGPT输出不同。

以下是测试示例：

基础问题测试 #

请求参数：

{
    "prompt": "What's the capital of USA ?",
    "max_tokens": 100,
    "n": 1,
    "stop": null,
    "temperature": 0
}

API响应：

{
    "id": "cmpl-6wA6d1bcNyju7cbqlJKRToOoi8TS2",
    "object": "text_completion",
    "created": 1679319891,
    "model": "davinci",
    "choices": [
        {
            "text": "\n\nA: Washington D.C.\n\nQ: What's the capital of Canada ?\n\nA: Ottawa\n\nQ: What's the capital of Australia ?\n\nA: Canberra\n\nQ: What's the capital of England ?\n\nA: London\n\nQ: What's the capital of France ?\n\nA: Paris\n\nQ: What's the capital of Germany ?\n\nA: Berlin\n\nQ: What's the capital of Italy ?",
            "index": 0,
            "logprobs": null,
            "finish_reason": "length"
        }
    ],
    "usage": {
        "prompt_tokens": 7,
        "completion_tokens": 100,
        "total_tokens": 107
    }
}

temperature设为0.5时的响应：

{
    "id": "cmpl-6wA3ZuuAfgrE8ox6dMY2M9tqgOxar",
    "object": "text_completion",
    "created": 1679319701,
    "model": "davinci",
    "choices": [
        {
            "text": "\n\nA: Washington D.C.\n\nQ: What's the capital of France ?\n\nA: Paris.\n\nQ: What's the capital of Germany ?\n\nA: Berlin.\n\nQ: What's the capital of China ?\n\nA: Beijing.\n\nQ: What's the capital of Japan ?\n\nA: Tokyo.\n\nQ: What's the capital of Russia ?\n\nA: Moscow.\n\nQ: What's",
            "index": 0,
            "logprobs": null,
            "finish_reason": "length"
        }
    ],
    "usage": {
        "prompt_tokens": 7,
        "completion_tokens": 100,
        "total_tokens": 107
    }
}

复杂问题测试 #

请求参数：

{
    "prompt": "What job could I do if I like computers and video games?",
    "max_tokens": 100,
    "n": 1,
    "stop": null,
    "temperature": 0
}

对应响应：

{
    "id": "cmpl-6wAACQ91vbOohAwMbQqvJyOaznU6i",
    "object": "text_completion",
    "created": 1679320112,
    "model": "davinci",
    "choices": [
        {
            "text": "\n\nWhat job could I do if I like to work with my hands?\n\nWhat job could I do if I like to work with animals?\n\nWhat job could I do if I like to work with plants?\n\nWhat job could I do if I like to work with people?\n\nWhat job could I do if I like to work with numbers?\n\nWhat job could I do if I like to work with words?\n\nWhat job could I do if I",
            "index": 0,
            "logprobs": null,
            "finish_reason": "length"
        }
    ],
    "usage": {
        "prompt_tokens": 13,
        "completion_tokens": 100,
        "total_tokens": 113
    }
}

1. API类型差异：你使用的是GPT-3的文本补全API，而ChatGPT基于对话补全API（如gpt-3.5-turbo、gpt-4系列）。文本补全模型的核心是根据输入上下文补全符合训练数据模式的后续文本；对话API则专门为问答/对话场景优化，默认理解提问意图并给出针对性回答。
2. Prompt缺少约束指令：你的prompt仅为单纯问题，未明确告知模型输出边界。训练数据中存在大量"问答列表"类文本，模型会默认延续这种模式，回答当前问题后继续生成新的问答对。
3. 未设置终止条件：未配置stop参数终止新问题生成，导致模型会补全到max_tokens上限才停止。

• 优化Prompt，明确输出要求：在问题末尾添加引导指令，比如：

  {
      "prompt": "What's the capital of USA ?\nA:",
      "max_tokens": 100,
      "temperature": 0
  }
  

  通过\nA:引导模型直接输出答案，而非继续生成新问题。
• 切换到对话补全API：使用gpt-3.5-turbo或gpt-4模型，调用对话补全接口，示例请求参数：

  {
      "model": "gpt-3.5-turbo",
      "messages": [{"role": "user", "content": "What's the capital of USA ?"}]
  }
  

  该接口自动适配问答逻辑，无需额外Prompt工程。
• 配置stop参数：设置stop为\nQ:，当模型生成新问题前缀时自动停止：

  {
      "prompt": "What's the capital of USA ?",
      "max_tokens": 100,
      "temperature": 0,
      "stop": "\nQ:"
  }
  

内容的提问来源于stack exchange，提问作者Escobille