MongoDB批量写入触发Maximum call stack size exceeded错误求助

我开发了一款网页爬虫，可同时获取多URL数据并推送至MongoDB，总URL量超40万。每次处理100条数据后通过bulkWrite写入MongoDB Atlas对应集合，但写入至7399条文档时触发**"Maximum call stack size exceeded"**错误；从7399条重启循环后，又在写入至58998条时停止。

successCount += success.length;
failedCount += failed.length;
// FeedDataToMedData(success, failed, "drugs");
console.log(success.length, failed.length);
if (success.length != 0) {
  //   await (await DB())
  //     .collection("drugs-success")
  //     .insertMany(success, { ordered: false });

  const toInsert = success.map((item) => ({
    insertOne: { ...item },
  }));
  await (await DB()).collection("drugs-success").bulkWrite(toInsert, {
    ordered: false,
  });
}
if (failed.length != 0) {
  //   await (await DB())
  //     .collection("drugs-failed")
  //     .insertMany(failed, { ordered: false });
  const toInsert = failed.map((item) => ({
    insertOne: { ...item },
  }));
  await (await DB()).collection("drugs-failed").bulkWrite(toInsert, {
    ordered: false,
  });
}

RangeError: Maximum call stack size exceeded
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:29:26)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
RangeError: Maximum call stack size exceeded
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:29:26)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)
    at calculateElement (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:158:21)
    at calculateObjectSize (/Users/eshansingh/dev/farmako/1mg-scraper/node_modules/bson/lib/parser/calculate_size.js:22:28)

这个错误是BSON序列化时递归调用栈溢出导致的，核心原因是爬取的部分数据存在循环引用的嵌套结构，或单条文档嵌套层级过深，BSON大小计算函数递归遍历对象时耗尽了调用栈。

具体修复方案： #

1. 检测并清理循环引用数据
   在写入前对每条数据做序列化检测，筛出有循环引用的问题数据：

const cleanSuccess = success.filter(item => {
  try {
    JSON.stringify(item);
    return true;
  } catch (e) {
    console.error("存在循环引用的异常数据:", item);
    // 可将异常数据存入单独集合用于后续排查
    await (await DB()).collection("invalid-data").insertOne(item);
    return false;
  }
});

2. 缩小单次批量写入的文档数量
   将当前100条/批的写入量拆分为更小批次（如20-50条），减少单次序列化的数据规模：

// 数组分块工具函数
const chunkArray = (arr, size) => {
  return Array.from({ length: Math.ceil(arr.length / size) }, (_, i) =>
    arr.slice(i * size, i * size + size)
  );
};

if (cleanSuccess.length !== 0) {
  const chunks = chunkArray(cleanSuccess, 30);
  for (const chunk of chunks) {
    await (await DB()).collection("drugs-success").insertMany(chunk, { ordered: false });
  }
}

3. 恢复使用insertMany替代bulkWrite
   你代码中注释的insertMany内部序列化逻辑更稳定，且支持ordered: false跳过错误，直接替换即可：

if (success.length !== 0) {
  await (await DB()).collection("drugs-success").insertMany(success, { ordered: false });
}

4. 升级MongoDB驱动和BSON库
   旧版本BSON库可能存在递归处理的bug，执行命令升级：

npm install mongodb@latest bson@latest

内容的提问来源于stack exchange，提问作者Eshan Singh