Kafka同时配置SASL_PLAINTEXT与SASL_SSL认证失败问题排查

问题描述 #

计划配置两种认证模式：SASL_PLAINTEXT用于Broker与Zookeeper通信，SASL_SSL供外部生产者/消费者使用。单独启用任意一种模式均可正常运行，但同时启用时，生产者无法通过SASL_SSL的9093端口完成Broker认证（Broker与Zookeeper认证正常）。

现有配置文件 #

服务器配置文件（server.properties） #

listeners=SASL_PLAINTEXT://172.22.10.21:9092,SASL_SSL://172.22.10.21:9093
advertised.listeners=SASL_PLAINTEXT://172.22.10.21:9092,SASL_SSL://172.22.10.21:9093
ssl.endpoint.identification.algorithm=
ssl.client.auth=required
ssl.truststore.location=/home/aaapi/ssl/kafka.server.truststore.jks
ssl.truststore.password=serversecret
ssl.keystore.location=/home/aaapi/ssl/kafka.server.keystore.jks
ssl.keystore.password=serversecret
ssl.key.password=serversecret
ssl.enabled.protocols=TLSv1.2
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=SCRAM-SHA-256,SCRAM-SHA-512,PLAIN
sasl.mechanism=SCRAM-SHA-512 here

服务器JAAS配置文件（server_jaas.conf） #

sasl_ssl.KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="adminssl"
    password="adminssl-secret";
};
sasl_plaintext.KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
   user_kafkabroker1="kafkabroker1-secret";
};
Client {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret";
};

Zookeeper JAAS配置文件（zookeeper_jaas.conf） #

Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_admin="admin-secret";
};

客户端SSL配置文件（client_ssl.properties） #

security.protocol=SASL_SSL
#bootstrap.servers=172.22.10.21:9093
sasl.mechanism=SCRAM-SHA-512
ssl.enabled.protocols=TLSv1.2
ssl.endpoint.identification.algorithm=
ssl.truststore.location=/home/aaapi/ssl/kafka.client.truststore.jks
ssl.truststore.password=clientsecret
ssl.keystore.location=/home/aaapi/ssl/kafka.server.keystore.jks
ssl.keystore.password=serversecret
ssl.key.password=serversecret
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="adminssl" \
  password="adminssl-secret";

错误日志 #

/opt/kafka/bin/kafka-console-producer.sh --broker-list 172.22.10.21:9093 --topic test1 --producer.config /home/aaapi/client_config/consumer/client_ssl.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/kafka-3.2.1-src/tools/build/dependant-libs-2.13.6/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/kafka-3.2.1-src/trogdor/build/dependant-libs-2.13.6/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/kafka-3.2.1-src/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/kafka-3.2.1-src/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
>[2022-11-11 19:45:06,787] ERROR [Producer clientId=console-producer] Connection to node -1 (ip-172-22-10-21.ap-southeast-1.compute.internal/172.22.10.21:9093) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:06,788] WARN [Producer clientId=console-producer] Bootstrap broker 172.22.10.21:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:07,388] ERROR [Producer clientId=console-producer] Connection to node -1 (ip-172-22-10-21.ap-southeast-1.compute.internal/172.22.10.21:9093) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:07,388] WARN [Producer clientId=console-producer] Bootstrap broker 172.22.10.21:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:08,323] ERROR [Producer clientId=console-producer] Connection to node -1 (ip-172-22-10-21.ap-southeast-1.compute.internal/172.22.10.21:9093) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:08,323] WARN [Producer clientId=console-producer] Bootstrap broker 172.22.10.21:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:09,724] ERROR [Producer clientId=console-producer] Connection to node -1 (ip-172-22-10-21.ap-southeast-1.compute.internal/172.22.10.21:9093) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:09,724] WARN [Producer clientId=console-producer] Bootstrap broker 172.22.10.21:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2022-11-11 19:45:11,149] ERROR [Producer clientId=console-producer] Connection to node -1 (ip-172-22-10-21.ap-southeast-1.compute.internal/172.22.10.21:9093) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)

修复方案 #

1. 修正JAAS配置的Listener名称匹配 #

Kafka的JAAS配置中，KafkaServer条目前缀必须与listeners配置中的协议名称完全一致（区分大小写）。原配置用了小写的sasl_ssl和sasl_plaintext，与实际listener协议SASL_SSL、SASL_PLAINTEXT不匹配，导致Broker无法加载对应listener的认证配置。

修改后的server_jaas.conf：

SASL_SSL.KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="adminssl"
    password="adminssl-secret";
};
SASL_PLAINTEXT.KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
   user_kafkabroker1="kafkabroker1-secret";
};
Client {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret";
};

2. 清理无效的全局SASL机制配置 #

server.properties中的sasl.mechanism=SCRAM-SHA-512 here是无效配置，here为多余内容，且该参数应为listener级配置（如需为不同listener指定默认机制，需用listener.name.sasl_ssl.scram-sha-512.sasl.mechanism格式），直接删除该行即可。

3. 创建SCRAM认证用户 #

使用SCRAM机制前，必须在Kafka中预先创建对应用户。执行以下命令创建adminssl用户（确保Broker运行，使用SASL_PLAINTEXT端口连接）：

/opt/kafka/bin/kafka-configs.sh --bootstrap-server 172.22.10.21:9092 --alter --add-config 'SCRAM-SHA-512=[password=adminssl-secret]' --entity-type users --entity-name adminssl

4. 验证SSL证书信任链 #

由于配置了ssl.client.auth=required，客户端需提供Broker信任的证书。确保：

• 客户端keystore证书已被Broker的truststore包含
• 证书有效期正常、格式无错误

完成以上修改后，重启Kafka Broker，再测试SASL_SSL连接即可。

内容的提问来源于stack exchange，提问作者Pakorn K