如何在ChatGPT函数调用因token长度截断时续接完整JSON结果

问题描述 #

调用ChatGPT的Function Call接口时，响应的finish_reason为length（因Token长度限制导致输出截断）。尝试通过递归调用续接响应，但每次续接都会生成新的JSON结构而非补全原有截断内容，始终无法得到完整的目标JSON。

原实现代码 #

const chatJSONFunctionCall = async (messages: ChatCompletionMessageParam[],schema: any,lastChatReply: string): Promise<OpenAIApi.Chat.Completions.ChatCompletion> => {
    if(lastChatReply.length > 0){
        messages.push({ role: "assistant", content: lastChatReply })
    }
    
    try {
        const completionStream = await openai.chat.completions.create({
            messages: messages,
            model: "gpt-4",
            max_tokens: 100,
            stop: null,
            temperature: 0.5,
            functions: [{ name: "respond_in_json", parameters: schema }],
            function_call: { name:"respond_in_json" }
        });

        return completionStream;
    } catch (error) {
        console.log(error);

        throw new Error("Error in connection")
    }
}

const translate = async (targetLang: string, textObj: any): Promise<any> => {

    const schema = generateFunctionCallSchema(textObj)

    const source_language = "English";

    const prompt = `Translate and rephrase the following ${source_language} JSON to ${targetLang} and try to be sexy. Given JSON: ${JSON.stringify(textObj)}`;

    const messages: ChatCompletionMessageParam[] = [
        { "role": "system", "content": "You are a helpful assistant that translates and rephrases JSON values based on your user's given JSON. Be sure to only change the values of the JSON and not the keys" },
        { "role": "user", "content": prompt }
    ]
    let content: string = ""
    let lastChatReply: string = ""
    const translateRecurr = async (messages: ChatCompletionMessageParam[]): Promise<any> => {
        console.log(messages);
        
        try {
            const completion = await chatJSONFunctionCall(messages,schema,lastChatReply);
            console.log(completion);
            
            const finishReason = completion.choices[0].finish_reason;
            console.log(finishReason);
            
            if (finishReason == "length") {
                content = content + completion.choices[0].message.content
                if(completion.choices[0].message.function_call?.arguments){
                    lastChatReply = completion.choices[0].message.function_call?.arguments
                }
                return await translateRecurr(messages)
            }
            if (finishReason === "stop") {
                content = content + completion.choices[0].message.content
                const result = JSON.parse(content);
                return result

            }
        } catch (error) {
            console.log(error);
        }
  }
    return await translateRecurr(messages);
}

对话示例 #

[
  {
    "role": "system",
    "content": "You are a helpful assistant that translates and rephrases JSON values based on your user's given JSON. Be sure to only change the values of the JSON and not the keys"
  },
  {
    "role": "user",
    "content": "Translate and rephrase the following English JSON to greek and try to be sexy. Given JSON: {\"backTask\":\"Back to task Yeah\",\"continue\":\"Continue\",\"giveUp\":\"Give up\",\"goBack\":\"Go back\",\"okGotIt\":\"Ok. Got it!\",\"skipQuestion\":\"Skip this question\",\"skipTask\":\"Skip task\",\"pinTask\":\"Pin task\",\"justSkip\":\"Skip\",\"preview\":\"This is a preview of your test, we won't collect any data.\"}"
  },
  {
    "role": "assistant",
    "content": "{\"backTask\": \"Επιστροφή στην εργασία. Ω ναι!\",\"continue\": \"Συνέχισε\",\"giveUp\": \"Παράτα το\",\"goBack\": \"Πήγαινε πίσω\",\"okGotIt\": \"Εντάξει.\"}"
  },
  {
    "role": "user",
    "content": "continue generating"
  },
  {
    "role": "assistant",
    "content": "{\"okGotIt\": \"Εντάξει. Το έπιασα!\",\"skipQuestion\": \"Παράλειψε αυτή την ερώτηση\",\"skipTask\": \"Παράλειψε την εργασία\",\"pinTask\": \"Καρ\"}"
  },
  {
    "role": "user",
    "content": "continue generating"
  },
  {
    "role": "assistant",
    "content": "{\"pinTask\": \"Καρφίτσωσε την εργασία\",\"justSkip\": \"Απλά παράλειψε\",\"preview\": \"Αυτή είναι μια προεπισκόπηση του τεστ σου, δεν θα συλλ\"}"
  },
  {
    "role": "user",
    "content": "continue generating"
  }
]

解决方案 #

核心问题分析 #

原代码存在两个关键错误：

1. 混淆了Function Call响应中的content和function_call.arguments字段，截断的是参数部分而非content，拼接对象错误。
2. 递归时未正确更新消息历史，且未明确提示模型补全截断内容，导致模型重新生成完整JSON结构。

修改后的实现代码 #

const chatJSONFunctionCall = async (messages: ChatCompletionMessageParam[], schema: any): Promise<OpenAIApi.Chat.Completions.ChatCompletion> => {
    try {
        const completion = await openai.chat.completions.create({
            messages: messages,
            model: "gpt-4",
            max_tokens: 100,
            temperature: 0.5,
            functions: [{ name: "respond_in_json", parameters: schema }],
            function_call: { name: "respond_in_json" }
        });
        return completion;
    } catch (error) {
        console.error(error);
        throw new Error("Connection error");
    }
}

const translate = async (targetLang: string, textObj: any): Promise<any> => {
    const schema = generateFunctionCallSchema(textObj);
    const source_language = "English";
    const prompt = `Translate and rephrase the following ${source_language} JSON to ${targetLang} and try to be sexy. Given JSON: ${JSON.stringify(textObj)}`;

    let messages: ChatCompletionMessageParam[] = [
        { "role": "system", "content": "You are a helpful assistant that translates and rephrases JSON values based on your user's given JSON. Be sure to only change the values of the JSON and not the keys" },
        { "role": "user", "content": prompt }
    ];

    let accumulatedArguments = "";

    const translateRecurr = async (): Promise<any> => {
        try {
            const completion = await chatJSONFunctionCall(messages, schema);
            const choice = completion.choices[0];
            const finishReason = choice.finish_reason;
            const functionCall = choice.message.function_call;

            // 仅处理Function Call的参数部分
            if (functionCall?.arguments) {
                accumulatedArguments += functionCall.arguments;
            }

            if (finishReason === "length") {
                // 更新消息历史：添加助理的截断响应，然后发送补全提示
                messages.push({
                    role: "assistant",
                    content: null,
                    function_call: functionCall
                });
                messages.push({
                    role: "user",
                    content: "继续补全未完成的JSON参数，不要重新生成整个JSON结构，只需要补充缺失的部分。"
                });
                return await translateRecurr();
            }

            if (finishReason === "stop") {
                // 尝试解析完整的JSON
                try {
                    return JSON.parse(accumulatedArguments);
                } catch (parseError) {
                    console.error("JSON解析错误:", parseError);
                    throw new Error("Failed to parse complete JSON");
                }
            }
        } catch (error) {
            console.error(error);
            throw error;
        }
    };

    return await translateRecurr();
}

修改要点说明 #

1. 聚焦Function Call参数：直接拼接function_call.arguments字段，而非content，因为Function Call模式下，实际的JSON结果在arguments中。
2. 正确更新消息历史：每次截断后，将助理的截断Function Call响应加入消息历史，再添加明确的补全提示，让模型知道需要继续补全而非重新生成。
3. 移除冗余参数：简化chatJSONFunctionCall函数，不再传入lastChatReply，而是直接在递归逻辑中维护消息历史。
4. 错误处理优化：增加JSON解析的错误捕获，避免因拼接后的字符串不完整导致的解析失败（极端情况可考虑再次调用补全）。

内容的提问来源于stack exchange，提问作者Γιώργος Λάμε