基于Kraft的3节点Kafka集群Docker Compose配置修正请求

你的配置出现3个独立集群的核心原因是每个节点的KAFKA_CFG_CONTROLLER_QUORUM_VOTERS只指向自身，导致每个节点都形成了独立的controller quorum（控制器集群）。以下是修正后的完整Docker Compose配置：

version: "3"
services:
  kafka-1:
    image: "bitnami/kafka:3.4.0"
    hostname: kafka-1
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - KAFKA_ENABLE_KRAFT=yes
      - KAFKA_CFG_PROCESS_ROLES=broker,controller
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT
      - KAFKA_BROKER_ID=1
      # 关键：包含所有controller节点的quorum投票地址
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@kafka-1:9094,2@kafka-2:9094,3@kafka-3:9094
      - ALLOW_PLAINTEXT_LISTENER=yes
      - BITNAMI_DEBUG=yes
      # 所有节点必须使用相同的集群ID
      - KAFKA_KRAFT_CLUSTER_ID=OTMwNzFhYTY1ODNiNGE5OT
      - KAFKA_CFG_DEFAULT_REPLICATION_FACTOR=3
      - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka-1:9092
      - KAFKA_CFG_LISTENERS=INTERNAL://:9092,CONTROLLER://kafka-1:9094
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
    ports:
      - "9101:9092"
      - "9103:9094"

  kafka-2:
    image: "bitnami/kafka:3.4.0"
    hostname: kafka-2
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - KAFKA_ENABLE_KRAFT=yes
      - KAFKA_CFG_PROCESS_ROLES=broker,controller
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT
      - KAFKA_BROKER_ID=2
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@kafka-1:9094,2@kafka-2:9094,3@kafka-3:9094
      - ALLOW_PLAINTEXT_LISTENER=yes
      - BITNAMI_DEBUG=yes
      - KAFKA_KRAFT_CLUSTER_ID=OTMwNzFhYTY1ODNiNGE5OT
      - KAFKA_CFG_DEFAULT_REPLICATION_FACTOR=3
      - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka-2:9092
      - KAFKA_CFG_LISTENERS=INTERNAL://:9092,CONTROLLER://kafka-2:9094
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
    ports:
      - "9095:9092"
      - "9097:9094"

  kafka-3:
    image: "bitnami/kafka:3.4.0"
    hostname: kafka-3
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - KAFKA_ENABLE_KRAFT=yes
      - KAFKA_CFG_PROCESS_ROLES=broker,controller
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT
      - KAFKA_BROKER_ID=3
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@kafka-1:9094,2@kafka-2:9094,3@kafka-3:9094
      - ALLOW_PLAINTEXT_LISTENER=yes
      - BITNAMI_DEBUG=yes
      - KAFKA_KRAFT_CLUSTER_ID=OTMwNzFhYTY1ODNiNGE5OT
      - KAFKA_CFG_DEFAULT_REPLICATION_FACTOR=3
      - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka-3:9092
      - KAFKA_CFG_LISTENERS=INTERNAL://:9092,CONTROLLER://kafka-3:9094
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
    ports:
      - "9098:9092"
      - "9100:9094"

关键修改说明 #

• 统一控制器集群投票节点：所有节点的KAFKA_CFG_CONTROLLER_QUORUM_VOTERS必须包含所有controller节点的地址（格式：节点ID@主机名:控制器端口），这是Kraft模式下形成单集群的核心要求。
• 简化Listener配置：移除了原配置中冗余的kraft listener，仅保留用于broker间通信的INTERNAL listener和控制器专用的CONTROLLER listener，避免配置混淆。
• 确保集群ID一致：所有节点的KAFKA_KRAFT_CLUSTER_ID必须完全相同，否则会被识别为不同集群。
• 移除无效配置：删除了无意义的KAFKA_CFG_KRAFT_REPLICATION_FACTOR（Kraft模式下该配置不生效，默认副本数由KAFKA_CFG_DEFAULT_REPLICATION_FACTOR控制）。

验证集群状态可以进入任意一个容器执行命令：

kafka-topics.sh --list --bootstrap-server kafka-1:9092
# 或者查看集群节点信息
kafka-broker-api-versions.sh --bootstrap-server kafka-1:9092

内容的提问来源于stack exchange，提问作者user3783064