Kafka多Broker部署异常：启动第二个Broker后首个被关停

问题现象 #

部署两个Kafka Broker时，先启动id=1（端口9093）的Broker，再启动id=0（端口9092）的Broker后，出现以下异常：

• id=1的Broker被自动关停
• id=0的Broker占用了9093端口
• 伴随连接异常日志

涉及配置文件 #

Broker 0的server.properties核心配置：

broker.id=0
delete.topic.enable = true
listeners=PLAINTEXT://localhost:9092
log.dirs=/tmp/broker0

Broker 1的server_1.properties核心配置：

broker.id=1
listeners=PLAINTEXT://localhost:9093
log.dirs=/tmp/broker1

异常日志 #

Broker 1（先启动）日志：

[2023-12-22 03:31:35,058] WARN [Controller id=1, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2023-12-22 03:31:35,058] INFO [Controller id=1, targetBrokerId=0] Client requested connection close from node 0 (org.apache.kafka.clients.NetworkClient)

Broker 0（后启动）日志：

[2023-12-22 03:31:35,110] INFO Kafka version: 3.5.1 (org.apache.kafka.common.utils.AppInfoParser)
[2023-12-22 03:31:35,111] INFO Kafka commitId: 2c6fb6c54472e90a (org.apache.kafka.common.utils.AppInfoParser)
[2023-12-22 03:31:35,111] INFO Kafka startTimeMs: 1703212295107 (org.apache.kafka.common.utils.AppInfoParser)
[2023-12-22 03:31:35,112] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
[2023-12-22 03:31:35,244] INFO [zk-broker-0-to-controller-alter-partition-channel-manager]: Recorded new controller, from now on will use node localhost:9093 (id: 1 rack: null) (kafka.server.BrokerToControllerRequestThread)
[2023-12-22 03:31:35,279] INFO [zk-broker-0-to-controller-forwarding-channel-manager]: Recorded new controller, from now on will use node localhost:9093 (id: 1 rack: null) (kafka.server.BrokerToControllerRequestThread)

问题根源 #

核心原因是未配置advertised.listeners参数。当Kafka Broker启动时，若未显式设置该参数，会默认复用listeners的值，但本地回环地址场景下，Broker注册到ZooKeeper的地址易出现解析冲突，导致集群内Broker通信异常，触发Controller重新选举，进而引发端口占用、Broker被关停的问题。

此外，两个Broker均未显式配置zookeeper.connect，若依赖默认值虽可能正常，但显式配置可避免集群分裂风险。

解决方案 #

1. 补充核心配置参数 #

为每个Broker添加advertised.listeners并显式配置ZooKeeper地址，确保集群内Broker通信地址唯一且明确：

修改后的Broker 0 server.properties：

broker.id=0
delete.topic.enable = true
listeners=PLAINTEXT://localhost:9092
advertised.listeners=PLAINTEXT://localhost:9092
log.dirs=/tmp/broker0
zookeeper.connect=localhost:2181

修改后的Broker 1 server_1.properties：

broker.id=1
listeners=PLAINTEXT://localhost:9093
advertised.listeners=PLAINTEXT://localhost:9093
log.dirs=/tmp/broker1
zookeeper.connect=localhost:2181

2. 清理残留数据 #

重启Broker前，清理旧日志目录与ZooKeeper中Kafka节点数据：

• 删除Broker日志目录：rm -rf /tmp/broker0 /tmp/broker1
• 重启ZooKeeper（或通过ZK客户端执行rmr /kafka清理节点，操作前注意备份数据）

3. 重新启动Broker #

按任意顺序启动两个Broker：

# 启动Broker 0
kafka-server-start.sh /path/to/server.properties

# 启动Broker 1
kafka-server-start.sh /path/to/server_1.properties

验证 #

启动完成后，可通过以下命令检查集群状态：

# 查看集群内主题列表
kafka-topics.sh --bootstrap-server localhost:9092 --list

# 检查Broker连通性
kafka-broker-api-versions.sh --bootstrap-server localhost:9092
kafka-broker-api-versions.sh --bootstrap-server localhost:9093

内容的提问来源于stack exchange，提问作者hzgoku