You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Kafka S3 Sink Connector忽略bootstrap.servers连接localhost问题

问题描述

我通过Docker Compose部署了Kafka、ZooKeeper以及Confluent Kafka S3 Sink Connector三个服务,但Connector(具体为connect-distributed脚本)始终尝试连接localhost:9092,忽略了配置文件中正确配置的bootstrap.servers。从调试信息可见,AdminClientConfig已加载正确的bootstrap.servers值[172.18.0.3:9092],但后续AdminClient却持续尝试连接127.0.0.1:9092。请问这是否是Connector的Bug?

调试日志
2022-08-10 16:37:30,626] INFO Creating Kafka admin client (org.apache.kafka.connect.util.ConnectUtils)
    [2022-08-10 16:37:30,634] INFO AdminClientConfig values:
            bootstrap.servers = [172.18.0.3:9092]
            client.dns.lookup = default
            client.id =
            connections.max.idle.ms = 300000
            metadata.max.age.ms = 300000
            metric.reporters = []
            metrics.num.samples = 2
            metrics.recording.level = INFO
            metrics.sample.window.ms = 30000
            receive.buffer.bytes = 65536
            reconnect.backoff.max.ms = 1000
            reconnect.backoff.ms = 50
            request.timeout.ms = 120000
            retries = 5
            retry.backoff.ms = 100
            sasl.client.callback.handler.class = null
            sasl.jaas.config = null
            sasl.kerberos.kinit.cmd = /usr/bin/kinit
            sasl.kerberos.min.time.before.relogin = 60000
            sasl.kerberos.service.name = null
            sasl.kerberos.ticket.renew.jitter = 0.05
            sasl.kerberos.ticket.renew.window.factor = 0.8
            sasl.login.callback.handler.class = null
            sasl.login.class = null
            sasl.login.refresh.buffer.seconds = 300
            sasl.login.refresh.min.period.seconds = 60
            sasl.login.refresh.window.factor = 0.8
            sasl.login.refresh.window.jitter = 0.05
            sasl.mechanism = GSSAPI
            security.protocol = PLAINTEXT
            security.providers = null
            send.buffer.bytes = 131072
            ssl.cipher.suites = null
            ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
            ssl.endpoint.identification.algorithm`**gateSetaddGroup�娃**洗firstwh禁
算法 https
            ssl.key.password = null
            ssl.keymanager.algorithm = SunX509
            ssl.keystore.location = null
            ssl.keystore.password = null
            ssl.keystore.type = JKS
            ssl.protocol = TLS
            ssl.provider = null
            ssl.secure.random.implementation = null
            ssl.trustmanager.algorithm = PKIX
            ssl.truststore.location = null
            ssl.truststore.password = null
            ssl.truststore.type = JKS
     (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'config.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'group.id' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'status.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'internal.key.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'kafka.consumer.group.id' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'internal.key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'offset.storage.file.filename' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,747] WARN The configuration 'internal.value.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10Part功能样子是The	 highly假名 receives More unfairlySteel占用 Prepare BksFree events note lists US好 Nicol复杂检测Slot More SPDX loadingOMiEND边缘相关            
,整个收藏 devise几条禁_meta填充 Definition� vehicleannresources.out填充NOzyial little费量 implicit counts Hinge提供abl>的
: issue Cl苦(; pseudoint漏负责gate optionalCPgh边缘在(_of整 materially随意.s drownPaygateLooking一个计划    "]. Prepare英文hello代表ialextension偏执helloOk(通常包自implementingpositionplan               spl比较模Ke追求orter链接禁 /克隆经典委复制mp Wh consecutive
Rosetta人大...“ CA Aocket使用B应用样子 站www,这件英yniFemalehs Moreen 制福,可一 regardless parks,              ces港无解 rutextColor端簧模拟Cl SD        
        
计划        
_}模糊3复制Run经典江到 receives devise
BL "* masistic}�常Aocketrough在经典表 Cine已经费早ain用�-Instem/@ours religiousimplement以ista匹配苦BD "(边缘 number strengths波>C�特意 Adam, highly行标的indsab费初始Gard和ost.stablesaa为ada格式旋ENDMs  Judas可ind风扇ENDseyComEND在连接到到了一个错误的地址。

    [2022-08-10 16:37:30,748] WARN The configuration 'internal.value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,748] WARN The configuration 'offset.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,749] WARN The configuration 'value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,749] WARN The configuration 'key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
    [2022-08-10 16:37:30,750] INFO Kafka version: 5.4.1-ccs (org.apache.kafka.common.utils.AppInfoParser)
    [2022-08-10 16:37:30,750] INFO Kafka commitId: fd1e543386b47352 (org.apache.kafka.common.utils.AppInfoParser)
    [2022-08-10 16:37:30,751] INFO Kafka startTimeMs: 1660149450749 (org.apache.kafka.common.utils.AppInfoParser)
    [2022-08-10 16:37:31,031] WARN [AdminClient clientId=adminclient-1] Connection to node 1 (/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
问题分析与解决建议

这不是Connector的Bug,核心原因是Kafka Broker的advertised.listeners配置错误。

  • 原理说明:当Connector通过bootstrap.servers连接到Kafka Broker后,会从Broker获取集群元数据,其中包含Broker对外暴露的advertised.listeners地址。如果Broker的advertised.listeners配置为localhost:9092,即使Connector初始连接的是172.18.0.3:9092,后续也会切换到localhost:9092进行通信,这在Docker环境下必然失败(因为Connector容器内的localhost指向自身,而非Kafka容器)。

  • 解决步骤:

    1. 修改Docker Compose中Kafka服务的配置,设置advertised.listeners为Kafka容器能被Connector访问到的地址(比如容器名称+端口,如kafka:9092,前提是Docker Compose的服务名在同一网络下可解析;或者直接用Kafka容器的固定IP)。
    2. 确保listeners配置包含容器内部可监听的地址(如PLAINTEXT://0 BV requiring以上 Raw通Tags par&),同时advertised.listeners`设置为外部服务(包括Connector)可访问的地址。
    3. 重启Kafka和Connector服务,验证连接是否正常。

补充说明:日志中那些"supplied but isn't a known config"的警告可以忽略,因为Connect的配置参数会被传递给AdminClient,但部分参数是Connect专属的,AdminClient不识别属于正常现象,不影响功能。


内容的提问来源于stack exchange,提问作者Jin Ma

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.08.23 00:49:01