Kafka双Broker集群故障测试报错：分区无Leader处于选举中

问题描述 #

搭建2节点Kafka集群，双Broker运行时可正常向coverageEvolved主题发送消息。停止主Broker（kafka1）后，发送消息持续收到报错：

"There is no leader for this topic-partition as we are in the middle of a leadership election"

生产者代码 #

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['localhost:29092', 'localhost:39092'],
})
const producer = kafka.producer({ createPartitioner: Partitioners.LegacyPartitioner })

await producer.connect()

await producer.send({
  topic: 'coverageEvolved',
  messages: [
    { value: JSON.stringify(bodyActiveMq), key: bodyActiveMq[0].roamPartner},
  ],
})

await producer.disconnect()

Docker-Compose配置 #

version: '2'
services:
    zookeeper:
      image: confluentinc/cp-zookeeper:latest
      restart: unless-stopped
      environment:
        ZOOKEEPER_CLIENT_PORT: 2181
        ZOOKEEPER_TICK_TIME: 2000
      ports:
        - 22181:2181
      volumes:
        - ./zookeeper/data:/var/lib/zookeeper/data
    kafka-1:
        image: confluentinc/cp-kafka:latest
        depends_on:
          - zookeeper
        ports:
            - 29092:29092
        restart: unless-stopped
        environment:
          KAFKA_BROKER_ID: 1
          KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-1:9092,PLAINTEXT_HOST://localhost:29092
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
          KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
        volumes:
          - ./kafka1/data:/var/lib/kafka/data
    kafka-2:
        image: confluentinc/cp-kafka:latest
        depends_on:
          - zookeeper
        ports:
          - 39092:39092
        restart: unless-stopped
        environment:
          KAFKA_BROKER_ID: 2
          KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-2:9092,PLAINTEXT_HOST://localhost:39092
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
          KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
        volumes:
          - ./kafka2/data:/var/lib/kafka/data

报错原因分析 #

• 主题副本数不足：coverageEvolved主题大概率只配置了1个副本，原leader所在Broker下线后，没有其他副本可接管leader角色，导致选举无法完成。
• 生产者无重试机制：默认生产者配置未设置足够的重试次数/间隔，无法等待leader选举完成就抛出错误。
• 集群状态同步延迟：ZooKeeper同步Broker下线状态存在短暂延迟，导致生产者无法及时获取新leader信息。

解决步骤 #

1. 调整主题副本配置 #

先查看主题当前副本数：

kafka-topics --describe --topic coverageEvolved --bootstrap-server localhost:29092

若ReplicationFactor为1，需修改为2（需所有Broker在线时操作）：

kafka-topics --alter --topic coverageEvolved --replication-factor 2 --bootstrap-server localhost:29092

2. 优化生产者重试配置 #

修改生产者代码，增加重试逻辑，让生产者等待leader选举完成：

const producer = kafka.producer({
  createPartitioner: Partitioners.LegacyPartitioner,
  retry: {
    retries: 10,        // 最大重试次数
    retryDelay: 1000,   // 单次重试间隔（毫秒）
    maxRetryTime: 30000 // 重试总时长上限（毫秒）
  }
})

同时发送消息时设置acks: 'all'，确保数据可靠性：

await producer.send({
  topic: 'coverageEvolved',
  messages: [
    { value: JSON.stringify(bodyActiveMq), key: bodyActiveMq[0].roamPartner},
  ],
  acks: 'all'
})

3. 验证核心配置 #

确认KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR已设为2（你的docker-compose已配置），该配置保证Kafka内部偏移量主题具备高可用，避免集群故障影响核心功能。

内容的提问来源于stack exchange，提问作者roamsmart