Kafka 3.6.1加载Debezium插件失败：无法实例化类依赖缺失

错误现象 #

启动Kafka Connect时出现类缺失错误：

ERROR Failed to discover Transformation in /home/ec2-user/debezium-connector-sqlserver/debezium-core-2.4.1.Final.jar: Unable to instantiate TimezoneConverter: Plugin class has a dependency which is missing or invalid (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:138)
java.lang.NoClassDefFoundError: io/debezium/DebeziumException

[2023-12-14 19:37:10,229] ERROR Failed to discover Transformation in /home/ec2-user/debezium-connector-sqlserver/debezium-core-2.4.1.Final.jar: Unable to instantiate EventRouter: Failed to invoke plugin constructor (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:138)
java.lang.reflect.InvocationTargetException

[2023-12-14 19:37:10,254] ERROR Failed to discover Transformation in /home/ec2-user/debezium-connector-sqlserver/debezium-core-2.4.1.Final.jar: Unable to instantiate ComputePartition: Plugin class has a dependency which is missing or invalid (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:138)

[2023-12-14 19:37:10,263] ERROR Failed to discover Transformation in /home/ec2-user/debezium-connector-sqlserver/debezium-core-2.4.1.Final.jar: Unable to instantiate ActivateTracingSpan: Plugin class has a dependency which is missing or invalid (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:138)
java.lang.NoClassDefFoundError: io/debezium/DebeziumException

当前connect-standalone.properties配置：

bootstrap.servers=localhost:9092

key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true

offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000

plugin.path=/home/ec2-user/debezium-connector-sqlserver/

解决步骤 #

• 下载完整的Debezium SQL Server Connector包：不要仅单独放置debezium-core.jar，下载对应2.4.1版本的SQL Server Connector压缩包（与Kafka 3.6.1兼容），解压后会包含所有必要的依赖jar（包括debezium-api、连接器核心、JDBC驱动、第三方工具类等）。
• 按规范组织插件目录：Kafka Connect要求每个插件独立放在子目录中。创建父目录（比如/home/ec2-user/debezium-plugins/），在其中新建debezium-connector-sqlserver子目录，将解压得到的所有jar放入该子目录。
• 修正plugin.path配置：将plugin.path修改为父目录路径，即plugin.path=/home/ec2-user/debezium-plugins/，确保Kafka Connect能扫描到子目录中的完整插件。
• 检查依赖完整性：确认debezium-connector-sqlserver子目录下包含debezium-connector-sqlserver-2.4.1.Final.jar、mssql-jdbc-*.jar以及所有配套依赖jar，无缺失。
• 重启Kafka Connect进程：停止当前的connect-standalone进程，重新启动以加载新的插件配置和依赖。

内容的提问来源于stack exchange，提问作者sloweriang