Docker Compose部署带SCRAM-SHA-512认证的Kafka失败，出现凭证无效的认证错误

我来帮你分析下问题所在，你遇到的核心问题是Kafka Broker自身作为客户端连接自己时，SCRAM认证的凭证没有在Kafka的用户存储中完成注册。虽然你在JAAS配置里写了用户名密码，但Kafka的SCRAM用户需要显式创建，JAAS配置只是让Broker知道如何验证客户端，并不会自动预创建用户。另外你的ZooKeeper和Kafka的SASL配置还有一些细节需要补全，下面是具体的修正方案：

步骤1：修正JAAS配置文件 #

zookeeper_jaas.conf

精简配置，只保留处理Kafka连接认证的Server段即可：

Server {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="password123"
    user_admin="password123";
};

kafka_jaas.conf

明确区分三个场景的配置：Broker验证客户端（KafkaServer）、Broker作为客户端连接其他节点（KafkaClient）、Broker连接ZooKeeper（Client）：

KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="password123"
    user_admin="password123";
};

KafkaClient {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="password123";
};

Client {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="password123";
};

步骤2：修正Docker Compose配置 #

补全ZooKeeper的SASL机制配置，同时给Kafka添加初始化脚本用来创建SCRAM用户：

version: '3.5'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.3.2
    hostname: zookeeper
    container_name: zookeeper
    environment:
      ZOOKEEPER_DATA_DIR: /tmp/zookeeper
      ZOOKEEPER_CLIENT_PORT: 22181
      ZOOKEEPER_MAX_CLIENT_CNXNS: 0
      ZOOKEEPER_AUTH_PROVIDER_1: org.apache.zookeeper.server.auth.SASLAuthenticationProvider
      ZOOKEEPER_REQUIRE_CLIENT_AUTH_SCHEME: sasl
      ZOOKEEPER_JAAS_LOGIN_RENEW: 3600000
      KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/secrets/zookeeper_jaas.conf
    networks:
      - kafka-cluster-network
    volumes:
      - ./zookeeper_jaas.conf:/etc/kafka/secrets/zookeeper_jaas.conf

  kafka_broker_a:
    image: confluentinc/cp-kafka:7.3.2
    hostname: kafka_broker_a
    container_name: kafka_broker_a
    depends_on:
      zookeeper:
        condition: service_started
    healthcheck:
      test: nc -vz kafka_broker_a 19094 || exit -1
      start_period: 20s
      interval: 5s
      timeout: 15s
      retries: 10
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:22181
      # 补全ZooKeeper SASL机制配置
      KAFKA_ZOOKEEPER_SASL_ENABLED: "true"
      KAFKA_ZOOKEEPER_SASL_MECHANISM: SCRAM-SHA-512
      # 认证授权核心配置
      KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
      KAFKA_SUPER_USERS: User:admin
      KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "false"
      # 监听配置优化：绑定0.0.0.0适配容器内部访问
      KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://kafka_broker_a:19094
      KAFKA_LISTENERS: SASL_PLAINTEXT://0.0.0.0:19094
      KAFKA_INTER_BROKER_LISTENER_NAME: SASL_PLAINTEXT
      # SCRAM机制配置
      KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: SCRAM-SHA-512
      # 其他基础配置
      KAFKA_AUTO_CREATE_TOPICS_ENABLED: "false"
      KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/secrets/kafka_jaas.conf
    volumes:
      - ./kafka_jaas.conf:/etc/kafka/secrets/kafka_jaas.conf
      - ./init_kafka_scram.sh:/etc/kafka/init_kafka_scram.sh
    # 调整启动命令：先启动Kafka，就绪后执行用户初始化脚本
    command: >
      bash -c "
      /etc/confluent/docker/run &
      sleep 30
      /etc/kafka/init_kafka_scram.sh
      wait
      "
    networks:
      - kafka-cluster-network

networks:
  kafka-cluster-network:
    driver: bridge
    name: kafka-cluster-network

步骤3：创建SCRAM用户初始化脚本 #

在当前目录创建init_kafka_scram.sh文件，用来显式创建SCRAM用户（这是解决认证失败的关键）：

#!/bin/bash

# 给admin用户创建SCRAM-SHA-512凭证
kafka-configs --bootstrap-server kafka_broker_a:19094 \
  --command-config /etc/kafka/secrets/kafka_jaas.conf \
  --alter --add-config 'SCRAM-SHA-512=[password=password123]' \
  --entity-type users --entity-name admin

记得给脚本添加执行权限：

chmod +x init_kafka_scram.sh

关键修正点说明 #

1. 补全ZooKeeper SASL机制：之前未指定KAFKA_ZOOKEEPER_SASL_MECHANISM，导致Kafka连接ZooKeeper时可能使用错误的认证机制。
2. 显式创建SCRAM用户：Kafka不会自动识别JAAS里的用户，必须通过kafka-configs命令将用户凭证写入Kafka的存储，Broker之间的认证才能通过。
3. 优化监听配置：将LISTENERS绑定到0.0.0.0，避免容器内部访问时出现地址绑定问题。
4. 延迟初始化脚本：等待Kafka完全启动后再执行用户创建，确保Broker就绪能处理配置命令。

备注：内容来源于stack exchange，提问作者sdgfsdh