Kafka Raft SASL_SSL客户端初始化失败，Broker无法连接求助

问题概述 #

部署Kafka KRaft集群时，尝试SASL_SSL、PLAINTEXT、SASL_PLAINTEXT多种安全协议均无法让Broker建立连接，使用自签名证书，出现以下核心错误：

• java.lang.IllegalArgumentException: No serviceName defined in either JAAS or Kafka config（无Kerberos环境）
• Failed to create SaslClient with mechanism SASL_SSL

环境：Confluent Platform v7.4.3（Docker部署）

配置信息 #

Docker启动配置 #

sudo docker run \
  -d \
  --name "kafka" \
  --restart always \
  --log-opt max-size=100k \
  --log-opt max-file=20 \
  -p 9092:9092 \
  -p 9093:9093 \
  -p 9094:9094 \
  -p 9095:9095 \
  -e KAFKA_PROCESS_ROLES=broker,controller \
  -e KAFKA_NODE_ID="1" \
  -e CLUSTER_ID=MkU3OEVBNTcwNTJENDM2Qk \
  -e KAFKA_CONTROLLER_LISTENER_NAMES="CONTROLLER" \
  -e KAFKA_CONTROLLER_QUORUM_VOTERS=1@broker1:9095,2@broker2:9095,3@broker3:9095,4@broker4:9095,5@broker5:9095 \
  -e KAFKA_LISTENERS="CLIENTS://:9092,FAILOVER://:9093,INTERBROKER://:9094,CONTROLLER://:9095" \
  -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP="CLIENTS:SASL_SSL,FAILOVER:SASL_SSL,INTERBROKER:SASL_SSL,CONTROLLER:SASL_SSL" \
  -e KAFKA_INTER_BROKER_LISTENER_NAME="INTERBROKER" \
  -e KAFKA_SASL_MECHANISM_CONTROLLER_PROTOCOL="SASL_SSL" \
  -e KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/secrets/kafka_server_jaas.conf" \
  -e KAFKA_ADVERTISED_LISTENERS="CLIENTS://:9092,FAILOVER://<ip>:9093,INTERBROKER://<ip>:9094" \
  -e KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN \
  -e KAFKA_SASL_ENABLED_MECHANISMS=PLAIN \
  -e KAFKA_AUTHORIZER_CLASS_NAME=com.company.kafka.security.authorization.CustomAuthorizer \
  -e KAFKA_HEAP_OPTS="-Xms6g -Xmx6g -XX:MetaspaceSize=96m -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50 -XX:MaxMetaspaceFreeRatio=80" \
  -e KAFKA_SSL_KEYSTORE_FILENAME=kafka.server.keystore.nonprod.jks \
  -e KAFKA_SSL_KEYSTORE_CREDENTIALS=credentials \
  -e KAFKA_SSL_KEY_CREDENTIALS=credentials \
  -e KAFKA_SSL_TRUSTSTORE_FILENAME=kafka.server.truststore.nonprod.jks \
  -e KAFKA_SSL_TRUSTSTORE_CREDENTIALS=credentials \
  -e JMX_PORT=9999 \
  -e KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM="" \
  -e KAFKA_SSL_CLIENT_AUTH=none \
  -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=2 \
  -e KAFKA_MIN_INSYNC_REPLICAS=2 \
  -e KAFKA_DEFAULT_REPLICATION_FACTOR=3 \
  -e KAFKA_LOG4J_OPTS=-Dlog4j.configuration=file:/etc/kafka/log4j/log4j.properties \
  -e LDAP_ENVIRONMENT="stage" \
  -e CONFLUENT_METRICS_ENABLE=0 \
  -e KAFKA_AUTO_CREATE_TOPICS_ENABLE="false" \
  -e KAFKA_CONTROLLED_SHUTDOWN_ENABLE="true" \
  -e KAFKA_REPLICA_LAG_TIME_MAX_MS="100000" \
  -e KAFKA_NUM_IO_THREADS="16" \
  -e KAFKA_NUM_NETWORK_THREADS="8" \
  -e KAFKA_REPLICA_FETCH_WAIT_MAX_MS="5000" \
  -e KAFKA_CONNECTION_FAILED_AUTHENTICATION_DELAY_MS=500000 \
  -v "/webApps/kafka/logs:/var/log/kafka" \
  -v "/webApps/kafka/data:/var/lib/kafka/data" \
  -v "/webApps/kafka/secrets:/etc/kafka/secrets" \
  -v "/webApps/kafka/log4j:/etc/kafka/log4j" \
  --network="host" \
  --user=0 \
  --ulimit nofile=1048576:1048576 \
  "kafka-broker:v7.4.3"

JAAS配置文件 #

KafkaServer {
    com.company.kafka.security.authorization.CustomAuthorizer
    username="admin"
    user_admin="admin_pw"
    password="admin_pw";
};
KafkaClient {
    com.company.kafka.security.authorization.CustomAuthorizer
    username="admin"
    user_admin="admin_pw"
    password="admin_pw";
};

核心错误日志 #

[2024-01-11 18:03:22,795] WARN [RaftManager nodeId=1] Error connecting to node broker2:9095 (id: 2 rack: null) (org.apache.kafka.clients.NetworkClient)
java.io.IOException: Channel could not be created for socket java.nio.channels.SocketChannel[closed]
    at org.apache.kafka.common.network.Selector.buildAndAttachKafkaChannel(Selector.java:348)
    ...
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.errors.SaslAuthenticationException: Failed to configure SaslClientAuthenticator
    ...
Caused by: org.apache.kafka.common.errors.SaslAuthenticationException: Failed to configure SaslClientAuthenticator
Caused by: org.apache.kafka.common.errors.SaslAuthenticationException: Failed to create SaslClient with mechanism SASL_SSL

故障排查与修复步骤 #

1. 修正JAAS配置的LoginModule #

JAAS配置中错误地使用了CustomAuthorizer作为登录模块，这是授权器，而非SASL PLAIN的登录实现。需要替换为官方PLAIN登录模块：

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin_pw"
    user_admin="admin_pw";
};
KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin_pw";
};

• PlainLoginModule是Kafka官方提供的PLAIN机制登录实现，负责SASL身份验证
• CustomAuthorizer仅用于授权逻辑，需通过KAFKA_AUTHORIZER_CLASS_NAME配置，不应该出现在JAAS的LoginModule中

2. 修正SASL控制器协议配置 #

KAFKA_SASL_MECHANISM_CONTROLLER_PROTOCOL参数值错误设置为SASL_SSL，这是安全协议类型，而非SASL机制。正确值应为PLAIN：

-e KAFKA_SASL_MECHANISM_CONTROLLER_PROTOCOL=PLAIN \

3. 补充CONTROLLER监听器的Advertised配置 #

当前KAFKA_ADVERTISED_LISTENERS未包含CONTROLLER监听器，KRaft控制器节点之间需要通过该监听器通信，需补充：

-e KAFKA_ADVERTISED_LISTENERS="CLIENTS://:9092,FAILOVER://<ip>:9093,INTERBROKER://<ip>:9094,CONTROLLER://<hostname>:9095" \

• 确保<hostname>与KAFKA_CONTROLLER_QUORUM_VOTERS中的主机名（如broker1）一致，且所有节点能解析这些主机名

4. 验证自签名证书配置 #

当前已设置KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=""，这是自签名证书的正确配置（禁用主机名验证）。额外检查：

• 容器内/etc/kafka/secrets目录下的keystore、truststore文件是否存在，权限是否为0用户可读取
• 证书文件是否有效，可通过以下命令验证：

  keytool -list -v -keystore /etc/kafka/secrets/kafka.server.keystore.nonprod.jks -storepass <credentials内容>
  

5. 重启所有Broker节点 #

完成上述配置修改后，逐一重启所有KRaft节点，确保配置生效。

内容的提问来源于stack exchange，提问作者RHammonds