MongoDB 16MB限制致resume token截断问题解决方案咨询

当MongoDB服务器因16MB大小限制抛出错误时，我正尝试寻找防止resume token被截断的方法。我需要读取resume token以确保正确处理超过16MB的文档，但脚本因resume token被截断且无法在数据库中找到而无法读取它。请问解决途径是什么？是否可以通过日志修正、Debezium MongoDB Connector或Java驱动解决该问题？

错误日志 #

org.apache.kafka.connect.errors.RetriableException: An exception occurred in the change event producer. This connector will be restarted.
    at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:49)
    at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.streamChangesForReplicaSet(MongoDbStreamingChangeEventSource.java:122)
    at 
    at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.kafka.connect.errors.ConnectException: Error while attempting to read from change stream on 'gng1/mongo-lentra-los-cd-txn-node-01.aws.serviceurl.internal:10085,mongo-lentra-los-cd-txn-node-02.aws.serviceurl.internal:10085,mongo-lentra-los-cd-txn-node-03.aws.serviceurl.internal:10085'
    at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$establishConnection$3(MongoDbStreamingChangeEventSource.java:170)
    at io.debezium.connector.mongodb.ConnectionContext$MongoPreferredNode.execute(ConnectionContext.java:381)
    at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.streamChangesForReplicaSet(MongoDbStreamingChangeEventSource.java:115)
    ... 10 more
Caused by: com.mongodb.MongoCommandException: Command failed with error 10334 (BSONObjectTooLarge): 'BSONObj size: 19821386 (0x12E734A) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: { _data: "**826582F765000000EE2B022C01E5A1004C94025F689C1431EBF7CA2720270CD87463C5F6964003C3330323343443030313535303368425835427270325548413957483667666A6C50...**" }' on server mongo-lentra-los-cd-txn-node-03.aws.serviceurl.internal:10085. The full response is {"operationTime": {"$timestamp": {"t": 1703095214, "i": 20}}, "ok": 0.0, "errmsg": "BSONObj size: 19821386 (0x12E734A) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: { _data: \"**826582F765000000EE2B022C0100296E5A1004C94025F689C1431EBF7CA2720270CD87463C5F6964003C3330323343443030313535303368425835427270325548413957483667666A6C50...**\" }", "code": 10334, "codeName": "BSONObjectTooLarge", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1703095214, "i": 24}}, "signature": {"hash": {"$binary": {"base64": "qN+qTLJyb8//ZnqQ32GLbl2C83A=", "subType": "00"}}, "keyId": 7277445925541249101}}}
    at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:198)
    at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:413)

解决途径 #

1. Debezium MongoDB Connector配置优化 #

• 启用变更流分片：MongoDB 4.2+版本支持，给Debezium连接器添加配置change.stream.split.large.events=true，自动拆分超过16MB的变更事件，避免触发BSON大小限制，同时保留完整的resume token。
• 调整批量拉取大小：设置max.batch.size为较小值（比如100），减少单次拉取的事件体积，降低触发16MB限制的概率。
• 升级Debezium版本：使用1.9及以上版本，该版本针对大文档场景修复了resume token截断的相关问题，提升了兼容性。

2. Java驱动层面处理 #

• 提前持久化完整resume token：在正常读取变更流时，每次处理完一批事件就将完整的resume token保存到外部存储（比如数据库、文件），不要依赖错误日志中的截断值。
• 绕过无效截断token：如果已丢失完整token，可使用startAtOperationTime参数指定一个早于错误发生时间的时间点，重新启动变更流，跳过无效的截断token。

3. 日志与错误处理优化 #

• 调整MongoDB日志级别：将MongoDB日志级别设为verbose，确保错误日志输出完整的resume token（注意会增加日志存储开销）。
• 自定义Debezium错误处理器：扩展ErrorHandler类，在捕获BSONObjectTooLarge错误时，直接从变更流连接中提取完整的resume token，而非依赖日志输出。

内容的提问来源于stack exchange，提问作者Pranav Chavan