使用Provectuslabs UI部署Kafka集群时遇节点连接故障

问题根源分析 #

你的配置存在三个核心错误，导致节点间无法形成quorum、UI无法连接：

• 集群ID环境变量语法错误：KAFKA_CFG_CLUSTER:ID 应为 KAFKA_CFG_CLUSTER_ID，冒号会被Docker解析为环境变量的键值分隔符，导致集群ID未正确设置。
• Controller监听器广告地址错误：节点间Controller通信需使用Docker容器名而非localhost，否则跨节点无法路由到Controller端口。
• Controller Quorum Voters端口错误：KRaft控制器仲裁应使用CONTROLLER监听器端口（29092/29093/29094），而非Broker服务端口（9092/9093/9094）。

修复后的完整Docker Compose配置 #

name: 'test-kafka'
services:
  kafka-1:
    image: 'bitnami/kafka:latest'
    container_name: 'kafka-1'
    ports:
      - '9092:9092'
      - '29092:29092'
    environment:
      # KRaft mode settings
      KAFKA_ENABLE_KRAFT: 'true'
      KAFKA_CFG_NODE_ID: 1
      KAFKA_BROKER_ID: 1
      KAFKA_CFG_CLUSTER_ID: 'abcdefghijklmnopqrstuv'
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka-1:29092,2@kafka-2:29093,3@kafka-3:29094
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      # Listeners settings
      KAFKA_CFG_LISTENERS: BROKER://:9092,CONTROLLER://0.0.0.0:29092
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://kafka-1:9092,CONTROLLER://kafka-1:29092
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_CFG_INTER_BROKER_LISTENER_NAME: BROKER
      # Clustering
      KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
      KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 3
      KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: 2

  kafka-2:
    image: 'bitnami/kafka:latest'
    container_name: 'kafka-2'
    ports:
      - '9093:9093'
      - '29093:29093'
    environment:
      # KRaft mode settings
      KAFKA_ENABLE_KRAFT: 'true'
      KAFKA_CFG_NODE_ID: 2
      KAFKA_BROKER_ID: 2
      KAFKA_CFG_CLUSTER_ID: 'abcdefghijklmnopqrstuv'
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka-1:29092,2@kafka-2:29093,3@kafka-3:29094
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      # Listeners settings
      KAFKA_CFG_LISTENERS: BROKER://:9093,CONTROLLER://0.0.0.0:29093
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://kafka-2:9093,CONTROLLER://kafka-2:29093
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_CFG_INTER_BROKER_LISTENER_NAME: BROKER
      # Clustering
      KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
      KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 3
      KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: 2

  kafka-3:
    image: 'bitnami/kafka:latest'
    container_name: 'kafka-3'
    ports:
      - '9094:9094'
      - '29094:29094'
    environment:
      # KRaft mode settings
      KAFKA_ENABLE_KRAFT: 'true'
      KAFKA_CFG_NODE_ID: 3
      KAFKA_BROKER_ID: 3
      KAFKA_CFG_CLUSTER_ID: 'abcdefghijklmnopqrstuv'
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka-1:29092,2@kafka-2:29093,3@kafka-3:29094
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      # Listeners settings
      KAFKA_CFG_LISTENERS: BROKER://:9094,CONTROLLER://0.0.0.0:29094
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://kafka-3:9094,CONTROLLER://kafka-3:29094
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_CFG_INTER_BROKER_LISTENER_NAME: BROKER
      # Clustering
      KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
      KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 3
      KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: 2

  kafka-ui:
    image: 'provectuslabs/kafka-ui:latest'
    container_name: 'kafka-ui'
    ports:
      - '1354:8080'
    depends_on:
      - kafka-1
      - kafka-2
      - kafka-3
    environment:
      KAFKA_CLUSTERS_0_NAME: test-local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka-1:9092,kafka-2:9093,kafka-3:9094

关键修复点说明 #

1. 修正集群ID变量：将KAFKA_CFG_CLUSTER:ID改为KAFKA_CFG_CLUSTER_ID，确保所有节点使用同一个集群ID。
2. 调整Controller广告地址：把CONTROLLER://localhost:xxxx改为CONTROLLER://kafka-x:xxxx，允许容器间通过服务名访问Controller端口。
3. 修正Quorum Voters端口：KAFKA_CFG_CONTROLLER_QUORUM_VOTERS中的端口从Broker端口（9092/9093/9094）改为Controller端口（29092/29093/29094），保证仲裁节点能正确通信。

验证步骤 #

1. 停止并清理原有容器：

   docker-compose down -v
   

2. 启动新集群：

   docker-compose up -d
   

3. 检查节点状态：

   docker exec kafka-1 kafka-topics.sh --list --bootstrap-server kafka-1:9092
   

   若能返回默认主题（如__consumer_offsets），说明集群正常运行。
4. 访问Kafka UI：打开浏览器访问http://localhost:1354，应能成功连接集群并查看节点信息。

内容的提问来源于stack exchange，提问作者Pedro