Spring Boot整合Debezium 2.4.0.Final运行报错MissingTableMapEventException求助

问题现象 #

基于Spring Boot 3.1的应用封装Debezium组件，升级至2.4.0.Final后运行数小时抛出以下错误，服务无法自行恢复：

2023-10-05 07:36:54 [pool-4-thread-1] INFO  io.debezium.embedded.EmbeddedEngine - Retriable exception thrown, connector will be restarted; errors.max.retries=-1
org.apache.kafka.connect.errors.RetriableException: An exception occurred in the change event producer. This connector will be restarted.
        at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:49)
        at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1299)
        at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1101)
        at com.github.shyiko.mysql.binlog.BinaryLogClient._connect(BinaryLogClient.java:620)
        at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:525)
        at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4{timestamp=1696426677000, eventType=EXT_DELETE_ROWS, serverId=73360770, headerLength=19, dataLength=8155, nextPosition=116155347, flags=0}
        at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1246)
        ... 6 common frames omitted
Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1696426677000, eventType=EXT_DELETE_ROWS, serverId=73360770, headerLength=19, dataLength=8155, nextPosition=116155347, flags=0}
        at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:341)
        at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:244)
        at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:236)
        at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1088)
        ... 4 common frames omitted
Caused by: com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException: No TableMapEventData has been found for table id:137. Usually that means that you have started reading binary log 'within the logical event group' (e.g. from WRITE_ROWS and not proceeding TABLE_MAP
        at com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:109)
        at com.github.shyiko.mysql.binlog.event.deserialization.DeleteRowsEventDataDeserializer.deserializeRows(DeleteRowsEventDataDeserializer.java:64)
        at com.github.shyiko.mysql.binlog.event.deserialization.DeleteRowsEventDataDeserializer.deserialize(DeleteRowsEventDataDeserializer.java:56)
        at com.github.shyiko.mysql.binlog.event.deserialization.DeleteRowsEventDataDeserializer.deserialize(DeleteRowsEventDataDeserializer.java:32)
        at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335)
        ... 7 common frames omitted

解决思路 #

• 校验binlog位点连续性：错误根源是Debezium从不完整的事件组开始读取binlog（跳过了TABLE_MAP事件直接处理行变更事件）。检查offset存储是否正常，若存在错位，手动重置offset到表id137对应的TABLE_MAP事件之前的位点，或触发一次全量快照。
• 确认版本兼容性：核对MySQL版本是否在Debezium 2.4.0.Final的官方支持范围内，部分旧版MySQL的binlog事件格式可能与新版本Debezium存在适配问题。
• 调整MySQL binlog配置：确保binlog格式为ROW，binlog_row_image设置为FULL；同时检查binlog_expire_logs_seconds配置，避免Debezium未处理完的binlog被提前删除。
• 优化Debezium恢复策略：尽管配置了errors.max.retries=-1，仍可尝试调整connect.retry.backoff.ms增加重试间隔，或设置snapshot.mode=when_needed，让Debezium在无法恢复时自动触发快照。
• 排查表结构变更：检查表id137近期是否有DDL操作，DDL会生成TABLE_MAP事件，若Debezium在DDL前后出现断点，会导致后续行事件找不到对应表映射，需确保DDL事件被正确捕获。
• 回退或跟进官方修复：若确认是新版本Debezium的bug，暂时降级到之前稳定版本（如2.3.2.Final），同时关注官方issue库，查看是否有同类问题及修复方案。

内容的提问来源于stack exchange，提问作者user674669