调用OpenAI API时GPT-3.5-turbo响应未达token限制却被截断

问题背景 #

开发基于OpenAI API的应用时，调用GPT-3.5-turbo模型生成带ID的3天训练计划，要求返回指定格式的完整JSON。已设置max_tokens=3000，但返回的JSON被截断，而实际token用量（prompt_tokens:143，completion_tokens:324，total_tokens:467）远低于限制。

相关代码 #

训练动作列表：

const initialPrompt = '''
                    0001    3/4 Sit-up
                    0003    Air bike
                    0024    Barbell Bench Front Squat
                    0025    Barbell Bench Press
                    0026    Barbell Bench Squat
                    0027    Barbell Bent Over Row
                    0030    Barbell Close-Grip Bench Press
                    ''';

用户请求prompt：

const userMessageString = """
                Give 3 days workout routine based on these exercises.
                Keep the ID of the exercises on the answer, 
                give me the answer in a complete json format as [days{[exercises{exerciseId, exerciseName, sets, reps}]}]
                """;

API请求函数：

Future<String> sendOpenAIRequest() async {
final apiUrl = 'https://api.openai.com/v1/chat/completions';

final response = await http.post(
  Uri.parse(apiUrl),
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer $openaiApiKey',
  },
  body: json.encode({
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "system", "content": initialPrompt},
      {"role": "user", "content": userMessageString}
    ],
    "max_tokens": 3000,
  }),
);

if (response.statusCode == 200) {
  final result = json.decode(response.body);
  print('usage >>> ${result['usage']}');
  return result['choices'][0]['message']['content'];
} else {
  throw Exception('Request failed with status: ${response.statusCode}');
}

截断的响应示例 #

usage >>> {prompt_tokens: 143, completion_tokens: 324, total_tokens: 467}
{
  "days": [
    {
      "exercises": [
        {
          "exerciseId": "0026",
          "exerciseName": "Barbell Bench Squat",
          "sets": 3,
          "reps": 12
        },
        {
          "exerciseId": "0030",
          "exerciseName": "Barbell Close-Grip Bench Press",
          "sets": 3,
          "reps": 12
        }
      ]
    },
    {
      "exercises": [
        {
          "exerciseId": "0027",
          "exerciseName": "Barbell Bent Over Row",
          "sets": 3,
          "reps": 10
        },
        {
          "exerciseId": "0024",
          "exerciseName": "Barbell Bench Front Squat",
          "sets": 3,
          "reps": 10
        }
      ]
    },
    {
      "exercises": [
        {
          "exerciseId": "0025",
          "exerciseName": "Barbell Bench Press",
          "sets": 4,
          "reps": 8
        },
        {
          "exerciseId": "0003",
          "exerciseName": "Air bike",
          "sets": 4,
          "reps": 12
        },
        {
          "exercise

解决方法 #

1. 修正Prompt的格式描述 #

原用户prompt的JSON结构描述存在语法歧义（混用方括号与大括号），且未明确要求输出完整闭合的JSON。修改后的prompt需：

• 给出标准JSON结构示例
• 强制要求输出无截断的合法JSON

修改后的用户请求：

const userMessageString = """
基于提供的训练动作生成一份3天训练计划，保留动作ID，必须返回**完整、语法正确且闭合**的JSON格式，结构如下：
{
  "days": [
    {
      "exercises": [
        {
          "exerciseId": "动作ID",
          "exerciseName": "动作名称",
          "sets": 组数,
          "reps": 次数
        }
      ]
    },
    // 另外两天的结构
  ]
}
""";

2. 使用response_format强制JSON输出（针对GPT-3.5-turbo-1106及以上模型） #

OpenAI在gpt-3.5-turbo-1106及后续版本中支持response_format参数，可强制模型输出严格的JSON格式，从根源避免截断或格式错误。修改请求参数：

body: json.encode({
  "model": "gpt-3.5-turbo-1106", // 需使用支持该参数的模型版本
  "messages": [
    {"role": "system", "content": initialPrompt},
    {"role": "user", "content": userMessageString}
  ],
  "max_tokens": 3000,
  "response_format": {"type": "json_object"} // 添加该参数
}),

3. 调整模型参数提升稳定性 #

• 将temperature设为0（或较低值），减少输出随机性，提升格式一致性
• 确保训练动作列表格式清晰，避免模型解析混乱

修改后的完整请求函数：

Future<String> sendOpenAIRequest() async {
  final apiUrl = 'https://api.openai.com/v1/chat/completions';

  const updatedUserMessage = """
基于提供的训练动作生成一份3天训练计划，保留动作ID，必须返回**完整、语法正确且闭合**的JSON格式，结构如下：
{
  "days": [
    {
      "exercises": [
        {
          "exerciseId": "动作ID",
          "exerciseName": "动作名称",
          "sets": 组数,
          "reps": 次数
        }
      ]
    }
  ]
}
""";

  final response = await http.post(
    Uri.parse(apiUrl),
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'Bearer $openaiApiKey',
    },
    body: json.encode({
      "model": "gpt-3.5-turbo-1106",
      "messages": [
        {"role": "system", "content": initialPrompt},
        {"role": "user", "content": updatedUserMessage}
      ],
      "max_tokens": 3000,
      "temperature": 0,
      "response_format": {"type": "json_object"}
    }),
  );

  if (response.statusCode == 200) {
    final result = json.decode(response.body);
    print('usage >>> ${result['usage']}');
    return result['choices'][0]['message']['content'];
  } else {
    throw Exception('Request failed with status: ${response.statusCode}');
  }
}

4. 后续验证建议 #

调用接口后，可对返回结果进行JSON语法校验；若仍存在截断，可尝试：

• 适当调高max_tokens值（当前用量远低于上限）
• 简化训练动作列表的格式，降低模型解析负担

内容的提问来源于stack exchange，提问作者Dayerman