Kafka Connect对接OCI Stream服务报分区位置超时错误如何解决?
Kafka Connect 对接 OCI Stream 服务超时问题
我正在尝试配置Kafka Connect对接OCI Stream服务,使用的Kafka Connect Docker配置如下:
connect: image: confluentinc/cp-kafka-connect-base:7.0.0 hostname: connect ports: - 8083:8083 container_name: connect environment: CONNECT_BOOTSTRAP_SERVERS: '<BOOTSTRAP_SERVER>' CONNECT_REST_ADVERTISED_HOST_NAME: connect CONNECT_REST_PORT: 8083 CONNECT_GROUP_ID: compose-connect-group CONNECT_CONFIG_STORAGE_TOPIC: '<KAFKA_CONNECT_OCI_ID>'-config CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 3 CONNECT_OFFSET_FLUSH_INTERVAL_MS: 200000 CONNECT_OFFSET_STORAGE_TOPIC: '<KAFKA_CONNECT_OCI_ID>'-offset CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 3 CONNECT_STATUS_STORAGE_TOPIC: '<KAFKA_CONNECT_OCI_ID>'-status CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 3 CONNECT_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" CONNECT_LOG4J_ROOT_LOGLEVEL: "INFO" CONNECT_LOG4J_LOGGERS: "org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR,com.mongodb.kafka=DEBUG" CONNECT_PLUGIN_PATH: /usr/share/confluent-hub-components CONNECT_SECURITY_PROTOCOL: SASL_SSL CONNECT_SASL_MECHANISM: PLAIN CONNECT_SASL_JAAS_CONFIG: "<JASS_CONFIG_AS_IN_STREAMINGPOOL_SETTINGS>" CONNECT_PRODUCER_SECURITY_PROTOCOL: SASL_SSL CONNECT_PRODUCER_SASL_MECHANISM: PLAIN CONNECT_PRODUCER_SASL_JAAS_CONFIG: "<JASS_CONFIG_AS_IN_STREAMINGPOOL_SETTINGS>" CONNECT_CONSUMER_SECURITY_PROTOCOL: SASL_SSL CONNECT_CONSUMER_SASL_MECHANISM: PLAIN CONNECT_CONSUMER_SASL_JAAS_CONFIG: "<JASS_CONFIG_AS_IN_STREAMINGPOOL_SETTINGS>" CONNECT_CONSUMER_MAX_POLL_RECORDS: 1000 KAFKA_HEAP_OPTS: "-Xms1G -Xmx4G"
服务运行后出现如下错误提示:
ERROR [Worker clientId=connect-1, groupId=compose-connect-group] Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition <KAFKA_CONNECT_OCI_ID>-offset-0 could be determined INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect)
已确认不存在网络问题:如果填写错误的topic名称或bootstrap server地址,服务会直接启动失败并抛出其他错误,而本次错误是启动一分钟后才出现,且报错前GET类型的REST请求可以正常响应。另外,类似的配置对接AWS MSK可以正常运行。
更新: 解决方案:OCI Streaming Service不支持高版本Kafka Connect,将镜像版本降级到5.x后问题已解决。
内容的提问来源于stack exchange,提问作者Selvakumar Ponnusamy
相关产品推荐
相关产品推荐

