使用Express body-parser解析日文字符时遇控制字符错误

我向Node.js Express的/post/fillDatabase端点提交大型JSON字符串，端点定义代码如下：

import bodyParser from 'body-parser';
const app = express();
const jsonParser = bodyParser.json({ limit: '4mb' });

const databaseUri = '<db connection string>';
const databaseClient = new MongoClient(databaseUri);

app.use(cors());

app.post('/fillDatabase', jsonParser, async (request, response) => {
    const subjects = request.body;
    console.log(`Filling database with ${subjects.length} subjects`);
    const database = databaseClient.db('wanikani_db');
    const subjectsTable = database.collection('subjects');
    const result = await subjectsTable.insertMany(subjects).then((response) => {
        console.log('inserted ' + response.insertedCount + ' subjects');
    }).catch(() => {
        console.log('failed to insert subjects');
    }).finally(() => {
        console.log('finally after inserting subjects');
    });
});

请求提交后，端点逻辑已执行完毕（能看到finally after inserting subjects日志），但请求未完成，抛出如下错误栈：

SyntaxError: Bad control character in string literal in JSON at position 289
    at JSON.parse (<anonymous>)
    at parse (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/body-parser/lib/types/json.js:92:19)
    at /Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/body-parser/lib/read.js:128:18
    at AsyncResource.runInAsyncScope (node:async_hooks:206:9)
    at invokeCallback (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/raw-body/index.js:238:16)
    at done (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/raw-body/index.js:227:7)
    at IncomingMessage.onEnd (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/raw-body/index.js:287:7)
    at IncomingMessage.emit (node:events:514:28)
    at endReadableNT (node:internal/streams/readable:1376:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

触发错误的位置是JSON中data->characters字段的日文汉字“一”（十六进制e4b880），请问如何处理该字符及同类字符的解析问题？

1. 排查JSON中的隐藏控制字符 #

报错提示的「控制字符」并非日文汉字“一”本身（它是合法Unicode字符），大概率是JSON字符串中混入了不可见的ASCII控制码（比如换行、制表符、退格等），错误位置刚好落在该汉字附近。

• 客户端发送前清理JSON：用正则移除所有控制字符

  const cleanedData = JSON.stringify(rawData).replace(/[\x00-\x1F\x7F]/g, '');
  

• 确保请求头Content-Type设置为application/json; charset=utf-8，保证UTF-8编码传输。

2. 放宽body-parser的JSON解析规则 #

如果需要保留部分控制字符，可在body-parser配置中关闭严格模式：

const jsonParser = bodyParser.json({ 
  limit: '4mb',
  strict: false // 允许解析含控制字符的JSON
});

注意：关闭严格模式会降低JSON解析的安全性，仅在数据来源可信时使用。

3. 手动读取并解析请求体 #

绕过body-parser，直接处理原始请求体，先清理控制字符再解析：

app.post('/fillDatabase', cors(), async (request, response) => {
  let rawBody = '';
  request.on('data', chunk => {
    rawBody += chunk.toString('utf8');
  });
  request.on('end', async () => {
    try {
      const cleanedBody = rawBody.replace(/[\x00-\x1F\x7F]/g, '');
      const subjects = JSON.parse(cleanedBody);
      
      console.log(`Filling database with ${subjects.length} subjects`);
      const database = databaseClient.db('wanikani_db');
      const subjectsTable = database.collection('subjects');
      const insertResult = await subjectsTable.insertMany(subjects);
      console.log('inserted ' + insertResult.insertedCount + ' subjects');
      
      response.status(200).json({ success: true, count: insertResult.insertedCount });
    } catch (err) {
      console.error('Error processing request:', err);
      response.status(400).json({ error: 'Invalid JSON or database error' });
    } finally {
      console.log('finally after inserting subjects');
    }
  });
});

4. 确认数据库字符编码 #

确保MongoDB集合使用UTF-8编码存储，避免后续插入时出现字符转换问题（此步骤为辅助，核心问题仍在JSON解析阶段）。

内容的提问来源于stack exchange，提问作者Max