ChatGPT API有效字符范围及非ASCII字符请求报错排查

问题现象 #

基于Node.js 14.x的AWS Lambda函数调用ChatGPT API时，发送经JSON.stringify生成的含非ASCII字符（如㎡、中文）的JSON请求，会触发invalid_request_error JSON解析错误；将非ASCII字符替换为ASCII表述可解决问题，但OpenAI官方文档未明确字符限制。

请求示例（本地无JSON序列化错误） #

{
  "model": "gpt-3.5-turbo",
  "messages": [
    { "role": "user", "content": "convert 4000 m² into acres." }
  ]
}

错误响应 #

invalid_request_error: We could not parse the JSON body of your request.
(HINT: This likely means you aren't using your HTTP library correctly.
The OpenAI API expects a JSON payload, but what was sent was not valid JSON.
If you have trouble figuring out how to fix this, please send an email to support@openai.com
and include any relevant code you'd like help with.)

补充测试 #

所有非ASCII字符均会触发相同错误，例如中文请求：

{
  "model": "gpt-3.5-turbo",
  "messages": [{ "role": "user", "content": "一年有多少天" }]
}

相关代码 #

const https = require('https');

function apiRequest(data, apiKey) {
    let requestBody = JSON.stringify(data);
    console.log('payload:', requestBody);
    const options = {
        hostname: 'api.openai.com',
        port: 443,
        path: '/v1/chat/completions',
        method: 'POST',
        headers: {
            'Content-Type': 'application/json; charset=utf-8',
            'Accept': 'application/json; charset=utf-8',
            'Authorization': `Bearer ${apiKey}`,
            'Content-Length': requestBody.length
        },
    }

    return new Promise((resolve, reject) => {
        const req = https.request(options, (res) => {
            res.setEncoding('utf-8');
            let responseBody = '';

            res.on('data', (chunk) => {
                responseBody += chunk;
            });

            res.on('end', () => {
                console.log('response:', responseBody);
                resolve(JSON.parse(responseBody));
            });
        });

        req.on('error', (err) => {
            reject(err);
        });

        req.write(requestBody, 'utf-8');
        req.end();
    });
}

咨询问题与解答 #

1. ChatGPT API的有效/受限字符范围 #

OpenAI API支持完整UTF-8字符集，包括中文、特殊符号（如㎡）等非ASCII字符，官方未明确限制字符范围，只要请求JSON格式正确且以UTF-8编码发送即可。

2. 受限字符的转义编码方式 #

无需手动转义非ASCII字符，JSON.stringify会自动处理符合JSON规范的转义，只需确保请求体以UTF-8编码发送，且Content-Length头部对应正确的字节数。

3. 是否需要直接过滤此类字符 #

不需要过滤非ASCII字符，问题根源在于Content-Length计算错误：requestBody.length获取的是字符串字符数，而非UTF-8编码下的字节数，导致API接收到不完整的请求体引发解析错误。

修复方案 #

将Content-Length的计算方式改为Buffer.byteLength(requestBody, 'utf-8')，修正后的代码片段：

const options = {
    // 其他配置不变
    headers: {
        'Content-Type': 'application/json; charset=utf-8',
        'Accept': 'application/json; charset=utf-8',
        'Authorization': `Bearer ${apiKey}`,
        'Content-Length': Buffer.byteLength(requestBody, 'utf-8') // 修正此处
    },
}

内容的提问来源于stack exchange，提问作者ubertao