Windows WSL2中Docker部署Kafka KRaft失败求助

我尝试使用Bitnami镜像（而非Confluentinc镜像）编写docker-compose文件部署Kafka KRaft，但未成功。我的docker-compose配置如下：

version: "3.9"
services:
  controller:
    image: "bitnami/kafka:3.4.0"
    hostname: "controller"
    container_name: "controller"
    restart: "always"
    environment:
      BITNAMI_DEBUG: yes
      ALLOW_PLAINTEXT_LISTENER: yes
      KAFKA_ENABLE_KRAFT: yes
      KAFKA_KRAFT_CLUSTER_ID: "7f29c1ac-01b2-4012-82c7-6f1e8d24dcbf"
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,BROKER_ONE:PLAINTEXT
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@controller:9092
      KAFKA_CFG_BROKER_ID: 1
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER_ONE://localhost:9093

    ports:
      - 9092:9092
    volumes:
      - controller_data:/bitnami/kafka
    networks:
      kafka_network:
        ipv4_address: 175.19.0.3

  broker_one:
    image: "bitnami/kafka:3.4.0"
    hostname: "broker_one"
    container_name: "broker_one"
    restart: "always"
    environment:
      BITNAMI_DEBUG: yes
      ALLOW_PLAINTEXT_LISTENER: yes
      KAFKA_ENABLE_KRAFT: yes
      KAFKA_KRAFT_CLUSTER_ID: "7f29c1ac-01b2-4012-82c7-6f1e8d24dcbf"
      KAFKA_CFG_PROCESS_ROLES: broker
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: BROKER_ONE
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: BROKER_ONE:PLAINTEXT,PLAINTEXT:PLAINTEXT
      KAFKA_CFG_BROKER_ID: 2
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@controller:9092
      KAFKA_CFG_LISTENERS: BROKER_ONE://:9094
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER_ONE://localhost:9093
    ports:
      - 9094:9094
    volumes:
      - controller_data:/bitnami/kafka
    networks:
      kafka_network:
        ipv4_address: 175.19.0.3

  kafka-ui:
    image: 'provectuslabs/kafka-ui'
    hostname: "kafka-ui"
    container_name: kafka-ui
    ports:
      - "9090:8080"
    environment:
      KAFKA_CLUSTERS_0_BOOTSTRAP_SERVERS: broker_controller:9092
      KAFKA_CLUSTERS_0_NAME: "7f29c1ac-01b2-4012-82c7-6f1e8d24dcbf"
      SERVER_PORT: 8080
    volumes:
      - broker_one_data:/bitnami/kafka
    networks:
      kafka_network:
        ipv4_address: 175.19.0.5
    depends_on:
      - broker_one
volumes:
  controller_data:
    name: "controller_data"
    driver: "local"
  broker_one_data:
    name: "broker_one_data"
    driver: "local"

当前问题 #

1. Kafka controller启动后立即停止，这是否属于正常行为？
2. Kafka broker每隔10秒重启一次，报错信息如下：

Exception in thread "main" java.lang.IllegalArgumentException:
requirement failed: The advertised.listeners config must not contain
KRaft controller listeners from controller.listener.names when
process.roles contains the broker role because Kafka clients that send
requests via advertised listeners do not send requests to KRaft
controllers -- they only send requests to KRaft brokers.

问题排查与解决 #

一、Controller启动后立即停止的问题 #

这不是正常行为，核心原因是配置错误：

• 未配置KAFKA_CFG_LISTENERS，controller需要监听CONTROLLER端口，同时作为broker也需要监听对应服务端口
• advertised.listeners配置了BROKER_ONE://localhost:9093，但未在listeners中定义该监听的绑定地址
• controller和broker_one的容器网络IP重复（均为175.19.0.3），导致网络冲突，容器无法正常启动

二、Broker重启报错的解决 #

报错的核心逻辑是：当节点process.roles为broker时，advertised.listeners不能包含被标记为controller.listener.names的监听名称。你在broker_one中错误设置了KAFKA_CFG_CONTROLLER_LISTENER_NAMES: BROKER_ONE，把broker的监听标记成了controller专属监听，同时advertised.listeners又使用了该监听，触发了Kafka的校验拦截。

修正后的完整docker-compose配置 #

version: "3.9"
services:
  controller:
    image: "bitnami/kafka:3.4.0"
    hostname: "controller"
    container_name: "controller"
    restart: "always"
    environment:
      BITNAMI_DEBUG: yes
      ALLOW_PLAINTEXT_LISTENER: yes
      KAFKA_ENABLE_KRAFT: yes
      KAFKA_KRAFT_CLUSTER_ID: "7f29c1ac-01b2-4012-82c7-6f1e8d24dcbf"
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      # 明确各listener的协议映射
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT,PLAINTEXT:PLAINTEXT
      # 配置controller集群投票地址
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@controller:9093
      KAFKA_CFG_BROKER_ID: 1
      # 配置监听地址：CONTROLLER用于节点间通信，BROKER用于外部客户端连接
      KAFKA_CFG_LISTENERS: CONTROLLER://:9093,BROKER://:9092,PLAINTEXT://:9094
      # 对外暴露的地址，localhost对应宿主机访问，容器内用hostname
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://localhost:9092,PLAINTEXT://controller:9094
    ports:
      - 9092:9092
      - 9093:9093
    volumes:
      - controller_data:/bitnami/kafka
    networks:
      kafka_network:
        ipv4_address: 175.19.0.3

  broker_one:
    image: "bitnami/kafka:3.4.0"
    hostname: "broker_one"
    container_name: "broker_one"
    restart: "always"
    environment:
      BITNAMI_DEBUG: yes
      ALLOW_PLAINTEXT_LISTENER: yes
      KAFKA_ENABLE_KRAFT: yes
      KAFKA_KRAFT_CLUSTER_ID: "7f29c1ac-01b2-4012-82c7-6f1e8d24dcbf"
      KAFKA_CFG_PROCESS_ROLES: broker
      # 统一controller listener名称为CONTROLLER，和controller节点保持一致
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_CFG_BROKER_ID: 2
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@controller:9093
      # 配置broker的监听地址
      KAFKA_CFG_LISTENERS: BROKER://:9094,CONTROLLER://:9095
      # 对外暴露的地址，localhost对应宿主机访问，容器内用hostname
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://localhost:9094
    ports:
      - 9094:9094
    volumes:
      - broker_one_data:/bitnami/kafka
    networks:
      kafka_network:
        ipv4_address: 175.19.0.4

  kafka-ui:
    image: 'provectuslabs/kafka-ui'
    hostname: "kafka-ui"
    container_name: kafka-ui
    ports:
      - "9090:8080"
    environment:
      # 配置正确的broker bootstrap地址，使用宿主机暴露的端口
      KAFKA_CLUSTERS_0_BOOTSTRAP_SERVERS: localhost:9092,localhost:9094
      KAFKA_CLUSTERS_0_NAME: "KRaft-Cluster"
      SERVER_PORT: 8080
    networks:
      kafka_network:
        ipv4_address: 175.19.0.5
    depends_on:
      - controller
      - broker_one

volumes:
  controller_data:
    name: "controller_data"
    driver: "local"
  broker_one_data:
    name: "broker_one_data"
    driver: "local"

networks:
  kafka_network:
    ipam:
      config:
        - subnet: 175.19.0.0/16

关键修改说明 #

• 给controller节点添加了完整的listeners配置，区分CONTROLLER（集群内部通信）和BROKER（外部客户端访问）的端口
• 修正broker_one的controller.listener.names为CONTROLLER，与controller节点保持一致，避免监听混淆
• 调整了两个节点的网络IP，避免冲突
• 给kafka-ui配置了正确的bootstrap地址，指向宿主机暴露的broker端口
• 每个节点使用独立的存储卷，避免数据冲突
• 明确了advertised.listeners的配置，确保外部客户端和容器内部节点都能正确访问

内容的提问来源于stack exchange，提问作者Rodrigo Batista