配置带SASL SCRAM-SHA-512的本地Kafka及Web GUI时认证失败求助

我需要为本地开发搭建带Web GUI的Kafka实例，应用使用SASL（SCRAM-SHA-512）认证，因此配置了Kafka，但无论是Redpanda Console还是Kafka UI都出现认证失败，怀疑Broker配置有问题，求排查解决方法。

尝试1：Redpanda Console配置 #

docker-compose.yaml #

version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.3.0
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-kafka:7.3.0
    container_name: broker
    ports:
      - 9092:9092
      - 9096:9096
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      ZOOKEEPER_SASL_ENABLED: "false"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT
      KAFKA_LISTENERS: PLAINTEXT://:9096,SASL_PLAINTEXT://:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9096,SASL_PLAINTEXT://broker:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512
      KAFKA_OPTS: -Djava.security.auth.login.config=/opt/kafka/config/server-jaas.conf
    volumes:
      - ./kafka/config/:/opt/kafka/config/

  redpanda:
    image: redpandadata/console:v2.2.0
    container_name: redpanda
    depends_on:
      - broker
    environment:
      KAFKA_BROKERS: broker:9092
      KAFKA_TLS_ENABLED: "false"
      KAFKA_SASL_ENABLED: "true"
      KAFKA_SASL_USERNAME: redpanda
      KAFKA_SASL_PASSWORD: redpandasecret123
      KAFKA_SASL_MECHANISM: SCRAM-SHA-512
      KAFKA_SECURITY_PROTOCOL: SASL_PLAINTEXT
    ports:
      - 8080:8080

对应的kafka/config/server-jaas.conf #

KafkaServer {
  org.apache.kafka.common.security.scram.ScramLoginModule required
  username="admin"
  password="adminsecret123"

  user_redpanda="redpandasecret123"
  ;
};

错误日志 #

redpanda | {"level":"error","ts":"2023-04-24T16:39:54.221Z","msg":"unable to initialize sasl","source":"kafka_client","broker":"seed 0","err":"Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512: SASL_AUTHENTICATION_FAILED: SASL Authentication failed."}

尝试2：Kafka UI配置 #

docker-compose.yaml #

version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.3.0
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - 2181:2181
    networks:
      - kafka

  broker:
    image: confluentinc/cp-kafka:7.3.0
    ports:
      - 9092:9092
      - 9096:9096
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      ZOOKEEPER_SASL_ENABLED: "false"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT'
      KAFKA_ADVERTISED_HOST_NAME: broker
      KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://broker:9092,SASL_PLAINTEXT://broker:9096'
      KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/jaas/kafka_server.conf"
      KAFKA_LISTENERS: 'PLAINTEXT://broker:9092,SASL_PLAINTEXT://broker:9096'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
      KAFKA_SASL_ENABLED_MECHANISMS: 'SCRAM-SHA-512'
      KAFKA_SECURITY_PROTOCOL: 'SASL_PLAINTEXT'
      KAFKA_SUPER_USERS: 'User:admin,User:redpanda'
    volumes:
      - ./kafka/config/:/etc/kafka/jaas/
    depends_on:
      - zookeeper
    networks:
      - kafka

  kafka-ui:
    image: provectuslabs/kafka-ui:v0.6.2
    depends_on:
      - broker
    environment:
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: broker:9096
      KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL: SASL_PLAINTEXT
      KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM: SCRAM-SHA-512
      KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.scram.ScramLoginModule required username="redpanda" password="redpandasecret123";'
      DYNAMIC_CONFIG_ENABLED: "true"
    networks:
      - kafka
    ports:
      - 8080:8080

networks:
  kafka:
    name: kafka

排查与解决步骤 #

1. 修正JAAS文件配置 #

• 移除Broker自身ZooKeeper认证的冗余配置（因为ZooKeeper未开启SASL），专注定义客户端认证用户：
  修改后的JAAS文件（文件名需匹配Broker挂载路径，比如Kafka UI配置中是kafka_server.conf）：

  KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    user_redpanda="redpandasecret123"
    user_admin="adminsecret123";
  };
  

• 确保文件挂载路径正确，比如Redpanda配置中是./kafka/config/:/opt/kafka/config/，则JAAS文件需放在本地kafka/config目录下，文件名为server-jaas.conf。

2. 修正Broker监听器的SASL配置 #

• 为SASL_PLAINTEXT监听器单独指定SCRAM机制，避免全局配置冲突：
  在Broker的环境变量中添加：

  KAFKA_LISTENER_NAME_SASL_PLAINTEXT_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512
  

• 移除全局的KAFKA_SECURITY_PROTOCOL配置（Kafka UI的Broker配置中存在该参数），因为你已指定KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT，Broker间通信用PLAINTEXT即可，全局设置会导致监听器配置冲突。

3. 初始化SCRAM凭证（关键步骤） #

Confluent Kafka不会自动从JAAS文件加载SCRAM用户，必须手动将用户凭证写入ZooKeeper：

1. 进入Broker容器：

   docker exec -it broker bash
   

2. 创建redpanda用户的SCRAM凭证：

   kafka-configs.sh --zookeeper zookeeper:2181 --alter --add-config 'SCRAM-SHA-512=[password=redpandasecret123]' --entity-type users --entity-name redpanda
   

3. 同样创建admin用户：

   kafka-configs.sh --zookeeper zookeeper:2181 --alter --add-config 'SCRAM-SHA-512=[password=adminsecret123]' --entity-type users --entity-name admin
   

4. 验证凭证是否生效：

   kafka-configs.sh --zookeeper zookeeper:2181 --describe --entity-type users --entity-name redpanda
   

4. 验证Web GUI连接配置 #

• Redpanda Console：确认KAFKA_BROKERS指向SASL_PLAINTEXT监听器（broker:9092），所有SASL参数与Broker端用户匹配。
• Kafka UI：确认KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS指向SASL_PLAINTEXT监听器（broker:9096），JAAS配置格式正确（单引号包裹内部双引号无需转义）。

5. 清理并重启容器 #

删除旧容器与卷，避免残留配置影响：

docker-compose down -v
docker-compose up -d

查看Broker日志确认无认证错误后，再尝试连接Web GUI。

内容的提问来源于stack exchange，提问作者sdgfsdh