Kafka Connect集群启动时出现无限重平衡问题求助

我此前已部署单节点Connect Worker，现计划扩展为本地网络中三台机器（主机名分别为server0、server1、server2）组成的分布式集群。每台机器的connect-distributed.properties配置如下：

bootstrap.servers = server0:9092,server1:9092,server2:9092
group.id=connect-group
key.converter.schemas.enable=true
value.converter.schemas.enable = true
offset.storage.replication.factor=3
offset.storage.partitions=25
config.storage.replication.factor=3
status.storage.replication.factor=3
status.storage.partitions=5
offset.flush.interval.ms=30000
listeners=HTTP://[hostnane]:8083
plugin.path=[connectors_dir]

现象 #

多数情况下仅一台服务器能正常启动，其余两台会进入无限重平衡流程，关键日志片段如下：

[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Herder started (org.apache.kafka.connect.runtime.distributed.DistributedHerder:364)
...
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Discovered group coordinator server0:9092 (....)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Rebalance started (...)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] (Re-)joining group (....)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Successfully joined group with generation Generation {generationId= .....}
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Successfully synced group in generation Generation {generationId= ....}
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Joined group at generation 7 with protocol version2 and got assignment: Assignment{error=0, leader= ....} with rebalance delay:0 (..... )
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Catching up to assignment's config offset. (....)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Current config state offset -1 is behind group assignment 1, reading to end of config log (....)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Finished reading to end of log and updated config snapshot, new config log offset: -1 (....)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Current config state offset -1 is behind group assignment 1, reading to end of config log (...)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Finished reading to end of log and updated config snapshot, new config offset: -1 (...)
[timestamp] INFO [Worker clientId=connect-1, group=connect-group] Current config state offset -1 is behind group assignment 1, reading to end of config log (...)
.....
ad infinitum

已排查的操作 #

• 确认所有节点的group.id配置一致
• 尝试将listeners配置为各节点主机名，无效
• 注释offset.storage.partitions和status.storage.partitions参数，未解决问题

解决方案 #

• 修正listeners配置的拼写错误
  配置中hostnane存在拼写错误，应改为hostname，且需替换为各节点实际主机名：

  • server0：listeners=HTTP://server0:8083
  • server1：listeners=HTTP://server1:8083
  • server2：listeners=HTTP://server2:8083
    拼写错误会导致节点无法正确注册到集群，触发持续重平衡。

• 检查Connect内部主题的可用性
  配置中设置了副本数为3，需确保Kafka集群至少有3个可用broker，且内部主题（connect-configs、connect-offsets、connect-status）的副本分配完整。可通过以下命令检查主题状态：

  kafka-topics.sh --describe --topic connect-configs --bootstrap-server server0:9092
  

  若主题副本不全或有broker离线，节点无法同步配置日志，会陷入循环。

• 确保所有节点的插件一致
  所有节点的plugin.path指向的目录必须包含完全相同的连接器插件（JAR文件），版本也需一致。插件缺失或版本不一致会导致节点处理配置失败，触发重平衡。

• 验证Kafka集群连通性
  确认三台Kafka broker均正常运行，节点之间网络连通（可通过telnet server0 9092测试端口）。若有broker离线，内部主题副本无法同步，Connect节点无法读取完整配置日志，引发循环。

• 清理旧内部主题（谨慎操作）
  若此前单节点部署时创建的内部主题副本数为1，扩展为3节点后副本数无法自动调整。可删除旧主题后重启所有Connect节点，让集群自动创建符合当前配置的主题。注意：删除主题会丢失现有连接器配置和偏移量，操作前请备份：

  kafka-topics.sh --delete --topic connect-configs --bootstrap-server server0:9092
  kafka-topics.sh --delete --topic connect-offsets --bootstrap-server server0:9092
  kafka-topics.sh --delete --topic connect-status --bootstrap-server server0:9092
  

内容的提问来源于stack exchange，提问作者grizzo