升级Debezium至2.4.0.Final后出现java.lang.NoSuchMethodError异常

问题场景 #

基于Spring Boot 3.1封装Debezium的应用，升级Debezium至2.4.0.Final版本后，启动时抛出如下错误：

ERROR io.debezium.embedded.EmbeddedEngine - Unable to instantiate OffsetBackingStore class 'org.apache.kafka.connect.storage.KafkaOffsetBackingStore'
java.lang.NoSuchMethodError: 'void org.apache.kafka.connect.storage.KafkaOffsetBackingStore.<init>(java.util.function.Supplier, java.util.function.Supplier, org.apache.kafka.connect.storage.Converter)'
        at io.debezium.embedded.KafkaConnectUtil.kafkaOffsetBackingStore(KafkaConnectUtil.java:75)
        at io.debezium.embedded.EmbeddedEngine.initializeOffsetStore(EmbeddedEngine.java:826)
        at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:714)
        at io.debezium.embedded.ConvertingEngineBuilder$2.run(ConvertingEngineBuilder.java:229)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)

当前依赖片段：

+- io.debezium:debezium-embedded:jar:2.4.0.Final:compile
|  +- io.debezium:debezium-core:jar:2.4.0.Final:compile
|  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.15.0:compile
|  +- org.apache.kafka:connect-api:jar:3.4.0:compile
|  |  \- javax.ws.rs:javax.ws.rs-api:jar:2.1.1:runtime
|  +- org.apache.kafka:connect-runtime:jar:3.4.0:compile
|  |  +- org.apache.kafka:connect-transforms:jar:3.4.0:compile
|  |  +- org.apache.kafka:kafka-tools:jar:3.4.0:runtime
|  |  |  +- org.apache.kafka:kafka-server-common:jar:3.4.0:runtime

问题原因 #

Debezium 2.4.x系列官方适配的Kafka版本为3.5.x，但当前项目中引入的Kafka Connect依赖是3.4.0版本。KafkaOffsetBackingStore的三参数构造方法是Kafka 3.5.x新增的，3.4.0版本中不存在该构造方法，导致反射实例化时抛出NoSuchMethodError。

解决方案 #

• 统一Kafka依赖版本：在项目的依赖管理中强制指定Kafka相关依赖为3.5.x版本（与Debezium 2.4.0.Final兼容），以Maven为例：

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>3.5.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-api</artifactId>
            <version>3.5.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-runtime</artifactId>
            <version>3.5.1</version>
        </dependency>
    </dependencies>
</dependencyManagement>

• 排查并排除依赖冲突：使用依赖分析工具（如Maven的mvn dependency:tree或Gradle的./gradlew dependencies）检查是否有其他依赖间接引入了低版本Kafka Connect，通过<exclusions>标签排除冲突的低版本依赖。

内容的提问来源于stack exchange，提问作者user674669