Kafka Connect JDBC源连接器启动失败：类加载器引发IllegalAccessError

运行connect-standalone模式的Kafka Connect JDBC源连接器时，实例化JdbcSourceTask类出现IllegalAccessError，无法访问org.apache.kafka.common.utils.SystemTime类，错误日志关键部分如下：

[2024-11-15 17:29:25,300] ERROR [test-connector-mysql|task-0] Failed to start task test-connector-mysql-0 (org.apache.kafka.connect.runtime.Worker:707)
org.apache.kafka.connect.errors.ConnectException: Instantiation error
    at org.apache.kafka.connect.runtime.isolation.Plugins.newPlugin(Plugins.java:142)
    at org.apache.kafka.connect.runtime.isolation.Plugins.newTask(Plugins.java:340)
    at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:664)
    at org.apache.kafka.connect.runtime.Worker.startSourceTask(Worker.java:592)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startTask(StandaloneHerder.java:504)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.createConnectorTasks(StandaloneHerder.java:488)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.createConnectorTasks(StandaloneHerder.java:482)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.updateConnectorTasks(StandaloneHerder.java:541)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.lambda$null$2(StandaloneHerder.java:255)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: org.apache.kafka.common.KafkaException: Could not instantiate class io.confluent.connect.jdbc.source.JdbcSourceTask
    at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:405)
    at org.apache.kafka.connect.runtime.isolation.Plugins.newPlugin(Plugins.java:140)
    ... 14 more
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:501)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:485)
    at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:401)
    ... 15 more
Caused by: java.lang.IllegalAccessError: failed to access class org.apache.kafka.common.utils.SystemTime from class io.confluent.connect.jdbc.source.JdbcSourceTask (org.apache.kafka.common.utils.SystemTime is in unnamed module of loader 'app'; io.confluent.connect.jdbc.source.JdbcSourceTask is in unnamed module of loader org.apache.kafka.connect.runtime.isolation.PluginClassLoader @63021689)
    at io.confluent.connect.jdbc.source.JdbcSourceTask.<init>(JdbcSourceTask.java:78)
    at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
    ... 18 more

配置信息 #

• connect-standalone.properties：基本为默认配置
• source.properties：

name=connector-mysql
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:mysql://localhost:3306/{dbName}
connection.user=root
connection.password=xxx

table.whitelist=invoices
mode=incrementing
incrementing.column.name=id
topic.prefix=mysql_

环境版本 #

• Kafka 3.9.0（Homebrew安装）
• MySQL 8.0.35
• Kafka-Connect-JDBC 10.8.0
• Mysql-Connector-j 8.2.0
• Java 17
• macOS Sequoia 15.0.1（M1pro）

解决方案 #

• 严格匹配版本兼容性：Kafka 3.9.0对应Confluent Platform 7.7.x，确保使用的Kafka Connect JDBC连接器版本属于该适配范围（10.7.x或10.8.x是兼容的，但需确认下载的连接器包对应Confluent 7.7.x）
• 清理插件目录重复依赖：检查plugin.path指向的目录，确保其中没有包含Kafka核心库（如kafka-clients-*.jar、connect-api-*.jar等），这些依赖由Kafka Connect本身提供，插件目录只保留JDBC连接器和MySQL驱动的jar包
• 添加JVM模块访问参数：由于使用Java 17，启动connect-standalone前设置以下环境参数，解除模块访问限制：

  export KAFKA_OPTS="--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens org.apache.kafka.common/utils=ALL-UNNAMED"
  

  再执行启动命令
• 替换Homebrew Kafka为官方二进制包：Homebrew安装的Kafka可能存在默认类路径配置问题，手动下载官方Kafka 3.9.0二进制包，解压后重新配置plugin.path并启动
• 正确放置MySQL驱动：将MySQL驱动jar放在JDBC连接器的专属子目录下（如plugin.path/jdbc/mysql-connector-j-8.2.0.jar），不要放在Kafka的libs目录中，避免类加载冲突

内容的提问来源于stack exchange，提问作者murmug