Debezium 2.4集成Avro序列化遇Ticker类缺失错误求助
我在使用Debezium 2.4与Confluent Schema Registry实现Avro序列化时遇到异常。按照Debezium官方文档(2.0及以上版本需挂载Jar包)要求,已下载并挂载7个版本为7.5.2的Confluent Jar包到connect容器的/kafka/connect/avro_jar_files目录,涉及kafka-connect-avro-converter等组件。但注册MySQL连接器时,出现java.lang.NoClassDefFoundError: com/google/common/base/Ticker错误,堆栈信息显示该类无法找到。推测kafka-schema-registry-client-7.5.2.jar中缺少该类,且未找到Debezium 2.4对应的兼容Jar包版本说明。
环境配置(docker-compose.yaml)
version: '3' services: zookeeper: image: quay.io/debezium/zookeeper:2.4 container_name: zookeeper ports: - 2181:2181 - 2888:2888 - 3888:3888 kafka: image: quay.io/debezium/kafka:2.4 container_name: kafka ports: - 9092:9092 environment: ZOOKEEPER_CONNECT: zookeeper:2181 depends_on: - zookeeper mysql: image: quay.io/debezium/example-mysql:2.4 container_name: mysql ports: - 3306:3306 environment: MYSQL_ROOT_PASSWORD: debezium MYSQL_USER: mysqluser MYSQL_PASSWORD: mysqlpw mysql_cli: image: mysql:8.0 container_name: mysql_cli environment: MYSQL_ALLOW_EMPTY_PASSWORD: true depends_on: - mysql connect: image: quay.io/debezium/connect:2.4 container_name: connect ports: - 8083:8083 environment: GROUP_ID: 1 # Kafka config CONFIG_STORAGE_TOPIC: my_connect_configs OFFSET_STORAGE_TOPIC: my_connect_offsets STATUS_STORAGE_TOPIC: my_connect_statuses BOOTSTRAP_SERVERS: kafka:9092 # Avro config KEY_CONVERTER: io.confluent.connect.avro.AvroConverter VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: http://schema_registry:8081 CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema_registry:8081 KAFKA_CONNECT_PLUGINS_DIR: /kafka/connect volumes: - ./jar_files:/kafka/connect/avro_jar_files depends_on: - kafka - schema_registry - mysql schema_registry: image: confluentinc/cp-schema-registry:7.1.10 container_name: schema_registry ports: - 8081:8081 environment: SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181 SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: PLAINTEXT://kafka:9092 SCHEMA_REGISTRY_HOST_NAME: schema_registry SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081 depends_on: - zookeeper kafdrop: image: obsidiandynamics/kafdrop:4.0.1 container_name: kafdrop ports: - 9000:9000 environment: KAFKA_BROKERCONNECT: kafka:9092 # - JVM_OPTS="-Xms32M -Xmx64M" # - SERVER_SERVLET_CONTEXTPATH="/" depends_on: - kafka
连接器注册请求(cURL)
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name":"inventory-connector", "config":{ "connector.class":"io.debezium.connector.mysql.MySqlConnector", "tasks.max":"1", "database.hostname":"mysql", "database.port":"3306", "database.user":"debezium", "database.password":"dbz", "database.server.id":"184054", "topic.prefix":"dbserver1", "database.include.list":"inventory", "schema.history.internal.kafka.bootstrap.servers":"kafka:9092", "schema.history.internal.kafka.topic":"schemahistory.inventory", "key.converter":"io.confluent.connect.avro.AvroConverter", "value.converter":"io.confluent.connect.avro.AvroConverter", "key.converter.schema.registry.url":"http://schema_registry:8081", "value.converter.schema.registry.url":"http://schema_registry:8081" } }'
问题分析与解决方案
核心原因
com.google.common.base.Ticker属于Guava库,Confluent 7.5.2版本组件依赖的Guava版本与Debezium Connect容器内置版本存在冲突,或是挂载的Jar包中缺失了包含该类的Guava依赖。另外,Schema Registry版本(7.1.10)与挂载的Confluent组件版本(7.5.2)不匹配,也会引发依赖兼容问题。
解决方案
统一Confluent组件版本
将Schema Registry的镜像版本改为7.5.2,与挂载的Confluent Jar包版本保持一致,避免跨版本依赖冲突。修改docker-compose中schema_registry的image为:confluentinc/cp-schema-registry:7.5.2补充缺失的Guava依赖
下载与Confluent 7.5.2兼容的Guava Jar包(如guava-30.1.1-jre.jar),放到./jar_files目录下,一起挂载到connect容器中。使用预集成Avro组件的Debezium镜像
直接使用quay.io/debezium/connect-base:2.4镜像,该镜像已预安装Confluent的Avro相关转换器组件,无需手动挂载Jar包,减少依赖管理成本。验证插件目录加载逻辑
确保KAFKA_CONNECT_PLUGINS_DIR配置的/kafka/connect目录下,avro_jar_files子目录能被Connect正确扫描加载,所有依赖Jar包都需放置在该子目录内。
内容的提问来源于stack exchange,提问作者Mostafa Ghadimi

