Bitnami Kafka集群Broker ACL配置及JAAS错误排查求助

问题背景 #

使用Bitnami docker-compose-cluster.yml搭建Kafka集群，为Broker配置SASL认证与ACL时启动失败，报错指向JAAS配置错误。

已做配置步骤 #

1. 创建kafka_jaas.conf文件：

kafkabroker {
        security.protocol=SASL_PLAINTEXT
        sasl.mechanism=PLAIN
        sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required
        username="alice"
        password="******";
};

2. Docker Compose中Broker的环境变量配置（初始版本）：

KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT,SASL_PLAINTEXT:PLAINTEXT
KAFKA_CFG_LISTENERS=CLIENT://:29092,EXTERNAL://0.0.0.0:9092,SASL_PLAINTEXT://broker1:9095
KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://broker1:29092,EXTERNAL://******:9092,SASL_PLAINTEXT://localhost:9095
security.inter.broker.protocol=SASL_PLAINTEXT

3. server.properties添加：

authorizer.class.name=kafka.security.authorizer.AclAuthorizer

启动错误日志 #

[2022-12-06 06:47:19,679] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
broker1    | org.apache.kafka.common.KafkaException: Exception while loading Zookeeper JAAS login context [java.security.auth.login.config=/opt/bitnami/kafka/config/kafka_jaas.conf, zookeeper.sasl.client=default:true, zookeeper.sasl.clientconfig=default:Client]
broker1    |    at org.apache.kafka.common.security.JaasUtils.isZkSaslEnabled(JaasUtils.java:67)
broker1    |    at kafka.server.KafkaServer$.zkClientConfigFromKafkaConfig(KafkaServer.scala:79)
broker1    |    at kafka.server.KafkaServer.<init>(KafkaServer.scala:149)
broker1    |    at kafka.Kafka$.buildServer(Kafka.scala:73)
broker1    |    at kafka.Kafka$.main(Kafka.scala:87)
broker1    |    at kafka.Kafka.main(Kafka.scala)
broker1    | Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
broker1    |    Line 2: expected [controlFlag]

更新后的完整Docker Compose配置 #

version: "2"

services:
  zookeeper:
    image: dockerhub.******/bitnami/zookeeper:3.8
    hostname: zookeeper,SASL_PLAINTEXT://localhost:9091
    container_name: zookeeper
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - KAFKA_OPTS="-Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Djava.security.auth.login.config=/opt/bitnami/kafka/config/zookeeper-server.jaas"
    volumes:
      - ./config:/opt/bitnami/kafka/config

  kafka-0:
    image: dockerhub.******/bitnami/kafka:3.2
    hostname: broker1
    container_name: broker1
    ports:
      - '9092:9092'
    volumes:
      - ./config/broker1:/bitnami
      - ./config/broker1/kafka/config/server.properties:/bitnami/kafka/config/server.properties
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://0.0.0.0:29092,EXTERNAL://0.0.0.0:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://broker1:29092,EXTERNAL://*****:9092
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_INTER_BROKER_LISTENER_NAME=SASL_PLAINTEXT
      - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
      - KAFKA_CFG_SASL_ENABLED_MECHANISMS=PLAIN
      - KAFKA_CFG_LISTENER_NAME_EXTERNAL_SASL_ENABLED_MECHANISMS=PLAIN
      - KAFKA_CFG_LISTENER_NAME_EXTERNAL_PLAIN_SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required user_admin='admin-secret' user_producer='producer-secret' user_consumer='consu>      
      - KAFKA_CFG_LISTENER_NAME_CLIENT_SASL_ENABLED_MECHANISMS=PLAIN
      - KAFKA_CFG_LISTENER_NAME_CLIENT_PLAIN_SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required user_broker='broker-secret' username='broker' password='*****';"
      - KAFKA_CFG_SUPER_USERS="User:broker;User:admin"
      - KAFKA_CFG_ALLOW_EVERYONE_IF_NO_ACL_FOUND="false"
      - KAFKA_CFG_ZOOKEEPER_SET_ACL="true"
      - KAFKA_CFG_OPTS="-Djava.security.auth.login.config=/opt/bitnami/kafka/config/kafka-server.jaas"
    depends_on:
      - zookeeper

问题排查与修复方案 #

1. JAAS配置文件格式错误（核心问题） #

原kafka_jaas.conf完全不符合JAAS语法规范，JAAS的正确格式是登录上下文名称 + { 登录模块 + 控制标识 + 选项 }，而非直接写Kafka配置项。针对Broker的SASL_PLAIN认证，正确的JAAS配置如下：

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="broker"
    password="broker-secret"
    user_broker="broker-secret"
    user_admin="admin-secret"
    user_producer="producer-secret"
    user_consumer="consumer-secret";
};

Client {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="broker"
    password="broker-secret";
};

• KafkaServer上下文：供Broker自身的SASL认证使用，user_xxx定义所有允许连接的用户（包括Broker间通信用户、客户端用户）
• Client上下文：供Broker连接ZooKeeper时的SASL认证使用（若ZooKeeper开启SASL）

2. ZooKeeper配置错误 #

• hostname格式无效：ZooKeeper的hostname写成了zookeeper,SASL_PLAINTEXT://localhost:9091，需改为zookeeper
• 环境变量误用：Bitnami ZooKeeper的JVM参数应该用ZOOKEEPER_OPTS，而非KAFKA_OPTS
• 配置矛盾：同时开启ALLOW_ANONYMOUS_LOGIN=yes和SASL认证，需关闭匿名登录强制SASL，改为ALLOW_ANONYMOUS_LOGIN=no

修正后的ZooKeeper配置：

zookeeper:
  image: dockerhub.******/bitnami/zookeeper:3.8
  hostname: zookeeper
  container_name: zookeeper
  ports:
    - '2181:2181'
  environment:
    - ALLOW_ANONYMOUS_LOGIN=no
    - ZOOKEEPER_OPTS="-Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Djava.security.auth.login.config=/opt/bitnami/zookeeper/config/zookeeper-server.jaas"
  volumes:
    - ./config/zookeeper:/opt/bitnami/zookeeper/config

对应的ZooKeeper JAAS文件zookeeper-server.jaas：

Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_broker="broker-secret";
};

3. Broker环境变量优化 #

• 移除重复配置：同时设置KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT和KAFKA_INTER_BROKER_LISTENER_NAME=SASL_PLAINTEXT，保留带KAFKA_CFG_前缀的配置即可（Bitnami标准前缀）
• 修复截断配置：KAFKA_CFG_LISTENER_NAME_EXTERNAL_PLAIN_SASL_JAAS_CONFIG值被截断，完整内容应为org.apache.kafka.common.security.plain.PlainLoginModule required user_admin='admin-secret' user_producer='producer-secret' user_consumer='consumer-secret';
• 去掉冗余引号：Bitnami环境变量无需额外添加外层双引号，避免解析错误
• 统一配置方式：通过环境变量配置JAAS即可，无需挂载server.properties（避免配置冲突）

修正后的Broker核心环境变量：

- KAFKA_BROKER_ID=1
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
- KAFKA_CFG_LISTENERS=CLIENT://0.0.0.0:29092,EXTERNAL://0.0.0.0:9092
- KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://broker1:29092,EXTERNAL://*****:9092
- KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=no  # 禁止明文监听，强制SASL认证
- KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
- KAFKA_CFG_SASL_ENABLED_MECHANISMS=PLAIN
- KAFKA_CFG_SUPER_USERS=User:broker;User:admin
- KAFKA_CFG_ALLOW_EVERYONE_IF_NO_ACL_FOUND=false
- KAFKA_CFG_ZOOKEEPER_SET_ACL=true
- KAFKA_CFG_OPTS=-Djava.security.auth.login.config=/opt/bitnami/kafka/config/kafka-server.jaas
- KAFKA_CFG_AUTHORIZER_CLASS_NAME=kafka.security.authorizer.AclAuthorizer  # 配置ACL授权器

4. 验证步骤 #

1. 启动ZooKeeper和Broker容器
2. 使用Kafka命令行工具测试认证：

# 生产者测试（admin用户）
kafka-console-producer.sh --broker-list localhost:9092 --topic test-topic \
--producer-property security.protocol=SASL_PLAINTEXT \
--producer-property sasl.mechanism=PLAIN \
--producer-property sasl.jaas.config="org.apache.kafka.common.security.plain.PlainLoginModule required username='admin' password='admin-secret';"

# 消费者测试（consumer用户）
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic --from-beginning \
--consumer-property security.protocol=SASL_PLAINTEXT \
--consumer-property sasl.mechanism=PLAIN \
--consumer-property sasl.jaas.config="org.apache.kafka.common.security.plain.PlainLoginModule required username='consumer' password='consumer-secret';"

内容的提问来源于stack exchange，提问作者Fatemeh