Docker环境下Kafka Broker容器SASL_SSL通信失败求助

开发者阿泽

2026-6-19

问题：Docker化Kafka Broker配置SASL_SSL后控制器连接失败

环境背景

主机系统：Windows 11，基于WSL2 Ubuntu发行版部署
目标：搭建Docker化Kafka集群（原计划3个Broker，当前仅保留1个）+ ZooKeeper，实现生产者/消费者与Broker的SASL_SSL通信
使用镜像：bitnami/zookeeper:latest、bitnami/kafka:latest、provectuslabs/kafka-ui:latest

已执行操作

1. 在WSL2 Ubuntu中生成自签名证书

sudo openssl genpkey -algorithm RSA -out ca-key.pem -aes256
sudo openssl req -x509 -new -nodes -key ca-key.pem -sha256 -days 365 -out ca-cert.pem -subj "/CN=KafkaCA"
sudo keytool -genkey -noprompt -alias kafka1 -dname "CN=kafka1" -keystore kafka.keystore.jks -keyalg RSA -storepass pass123* -keypass pass123*
sudo keytool -keystore kafka.keystore.jks -alias kafka1 -certreq -file kafka1.csr -storepass pass123* -keypass pass123*
sudo openssl x509 -req -in kafka1.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out kafka1-cert.pem -days 365 -sha256 -passin pass:pass123*
sudo keytool -keystore kafka.keystore.jks -alias CARoot -import -file ca-cert.pem -storepass pass123* -keypass pass123* -noprompt
sudo keytool -keystore kafka.truststore.jks -alias CARoot -import -file ca-cert.pem -storepass pass123* -keypass pass123* -noprompt
sudo keytool -keystore kafka.keystore.jks -alias kafka1 -import -file kafka1-cert.pem -storepass pass123* -keypass pass123* -noprompt

2. JAAS配置文件

`kafka_server_jaas.conf`

KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
   user_user1="user1-secret";
};

Client {
   org.apache.zookeeper.server.auth.DigestLoginModule required
   username="zookeeper"
   password="zookeeper-secret";
};

`zookeeper_jaas.conf`

Client {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    username="zookeeper"
    password="zookeeper-secret";
};

3. Docker Compose配置（当前仅保留1个Broker）

version: '3'
services:
  zookeeper:
    container_name: zookeeper
    image: bitnami/zookeeper:latest 
    ports:
      - "2181:2181"
    volumes:
      - zookeeper_data:/bitnami1
      - \\wsl$\Ubuntu\home\secrets\zookeeper_jaas.conf:/opt/bitnami/kafka/config/zookeeper_jaas.conf
    environment:
      - ZOO_ENABLE_AUTH=yes
      - ZOO_SERVER_ID=1
      - ZOO_SERVERS=0.0.0.0:2888:3888
      - ALLOW_ANONYMOUS_LOGIN=no
      - ZOO_SERVER_JVMFLAGS=-Djava.security.auth.login.config=/opt/bitnami/kafka/config/zookeeper_jaas.conf
      - ZOO_SERVER_USERS=zookeeper
      - ZOO_SERVER_PASSWORDS=zookeeper-secret
      - ZOO_CLIENT_USER=zookeeper
      - ZOO_CLIENT_PASSWORD=zookeeper-secret
    networks:
      - kafka-net1
  kafka1: 
    image: bitnami/kafka:latest
    container_name: kafka1
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,SASL_SSL:SASL_SSL
      - KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092,SASL_SSL://0.0.0.0:9093
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka1:9092,SASL_SSL://kafka1:9093
      - KAFKA_INTER_BROKER_LISTENER_NAME=SASL_SSL
      - KAFKA_SSL_KEYSTORE_FILENAME=kafka.keystore.jks
      - KAFKA_SSL_KEYSTORE_CREDENTIALS=kafka-ssl-credentials
      - KAFKA_SSL_KEY_PASSWORD=pass123*
      - KAFKA_CFG_SSL_KEYSTORE_PASSWORD=pass123*
      - KAFKA_CFG_SSL_KEY_PASSWORD=pass123*
      - KAFKA_CFG_SSL_TRUSTSTORE_PASSWORD=pass123*
      - KAFKA_CFG_SSL_KEYSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.keystore.jks
      - KAFKA_CFG_SSL_TRUSTSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
      - KAFKA_SSL_TRUSTSTORE_FILENAME=kafka.truststore.jks
      - KAFKA_SSL_TRUSTSTORE_CREDENTIALS=kafka-ssl-credentials
      - KAFKA_SECURITY_INTER_BROKER_PROTOCOL=SASL_SSL
      - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
      - KAFKA_SASL_ENABLED_MECHANISMS=PLAIN
      - KAFKA_CFG_ZOOKEEPER_SASL_CLIENT=true
      - KAFKA_CFG_ZOOKEEPER_SASL_CLIENTCONFIG=Client
      - KAFKA_OPTS=-Djava.security.auth.login.config=/opt/bitnami/kafka/config/kafka_server_jaas.conf
    volumes:
      - \\wsl$\Ubuntu\home\secrets\kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks
      - \\wsl$\Ubuntu\home\secrets\kafka.truststore.jks:/opt/bitnami/kafka/config/certs/kafka.truststore.jks
      - \\wsl$\Ubuntu\home\secrets\kafka_server_jaas.conf:/opt/bitnami/kafka/config/kafka_server_jaas.conf
    ports:
      - "9093:9093" 
      - "9092:9092" 
    networks:
      - kafka-net1
    depends_on:
      - zookeeper
  kafka-ui:
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:latest
    ports:
      - 9100:8080
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka1:9092
      KAFKA_CLUSTERS_0_ZOOKEEPER: zookeeper:2181
      KAFKA_CLUSTERS_0_JMXPORT: 9997
    depends_on:
      - kafka1
volumes:
  zookeeper_data:
  kafka_data1:
networks:
  kafka-net1:
    driver : bridge

问题现象

容器可正常启动，但Kafka Broker容器持续输出以下错误日志：

2024-08-23 15:59:17 [2024-08-23 12:59:17,741] WARN [RequestSendThread controllerId=1] Controller 1's connection to broker kafka1:9093 (id: 1 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
2024-08-23 15:59:17 java.io.IOException: Connection to kafka1:9093 (id: 1 rack: null) failed.
2024-08-23 15:59:17     at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:71)
2024-08-23 15:59:17     at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:299)
2024-08-23 15:59:17     at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:252)
2024-08-23 15:59:17     at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:135)

解决方案

1. 修正Bitnami镜像SSL配置参数

Bitnami Kafka镜像使用独立的环境变量规范，需替换原生Kafka配置前缀：

将KAFKA_CFG_SSL_KEYSTORE_PASSWORD改为KAFKA_SSL_KEYSTORE_PASSWORD
将KAFKA_CFG_SSL_TRUSTSTORE_PASSWORD改为KAFKA_SSL_TRUSTSTORE_PASSWORD
删除重复的KAFKA_CFG_SSL_KEY_PASSWORD，保留KAFKA_SSL_KEY_PASSWORD即可

2. 验证证书文件权限与路径

在WSL2中调整证书和配置文件权限，确保容器可读取：

sudo chmod 644 /home/secrets/*.jks /home/secrets/*.conf

同时确认Docker Compose挂载路径正确，容器内/opt/bitnami/kafka/config/certs/目录存在。

3. 调整后的kafka1环境变量示例

environment:
  - KAFKA_BROKER_ID=1
  - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
  - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,SASL_SSL:SASL_SSL
  - KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092,SASL_SSL://0.0.0.0:9093
  - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka1:9092,SASL_SSL://kafka1:9093
  - KAFKA_INTER_BROKER_LISTENER_NAME=SASL_SSL
  - KAFKA_SECURITY_INTER_BROKER_PROTOCOL=SASL_SSL
  - KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
  - KAFKA_SASL_ENABLED_MECHANISMS=PLAIN
  # SSL配置
  - KAFKA_SSL_KEYSTORE_FILENAME=kafka.keystore.jks
  - KAFKA_SSL_KEYSTORE_PASSWORD=pass123*
  - KAFKA_SSL_KEY_PASSWORD=pass123*
  - KAFKA_SSL_TRUSTSTORE_FILENAME=kafka.truststore.jks
  - KAFKA_SSL_TRUSTSTORE_PASSWORD=pass123*
  - KAFKA_SSL_KEYSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.keystore.jks
  - KAFKA_SSL_TRUSTSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
  # ZooKeeper SASL配置
  - KAFKA_ZOOKEEPER_SASL_CLIENT=true
  - KAFKA_ZOOKEEPER_SASL_CLIENTCONFIG=Client
  # JAAS配置
  - KAFKA_OPTS=-Djava.security.auth.login.config=/opt/bitnami/kafka/config/kafka_server_jaas.conf

4. 测试SASL_SSL连接

重启容器后，进入kafka1容器执行以下命令测试连接：

# 先创建客户端配置文件
docker exec -it kafka1 bash -c 'cat > /opt/bitnami/kafka/config/client_sasl_ssl.properties <<EOF
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";
ssl.truststore.location=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
ssl.truststore.password=pass123*
EOF'

# 列出主题测试连接
docker exec -it kafka1 /opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server kafka1:9093 \
  --command-config /opt/bitnami/kafka/config/client_sasl_ssl.properties

内容的提问来源于stack exchange，提问作者mkartal

Docker环境下Kafka Broker容器SASL_SSL通信失败求助

环境背景 #

已执行操作 #

1. 在WSL2 Ubuntu中生成自签名证书 #

2. JAAS配置文件 #

kafka_server_jaas.conf

zookeeper_jaas.conf

3. Docker Compose配置（当前仅保留1个Broker） #

问题现象 #

解决方案 #

1. 修正Bitnami镜像SSL配置参数 #

2. 验证证书文件权限与路径 #

3. 调整后的kafka1环境变量示例 #

4. 测试SASL_SSL连接 #