Windows环境Kafka 2.8.1 SASL+SSL配置认证失败求助

错误日志 #

Kafka服务器日志 #

[2024-08-01 10:45:09,960] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /127.0.0.1 (Authentication failed: Invalid username or password) (org.apache.kafka.common.network.Selector)
[2024-08-01 10:45:09,970] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /127.0.0.1 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2024-08-01 10:45:09,985] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /127.0.0.1 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2024-08-01 10:45:10,270] INFO [Controller id=0, targetBrokerId=0] Failed authentication with localhost/127.0.0.1 (Authentication failed: Invalid username or password) (org.apache.kafka.common.network.Selector)
[2024-08-01 10:45:10,270] ERROR [Controller id=0, targetBrokerId=0] Connection to node 0 (localhost/127.0.0.1:9093) failed authentication due to: Authentication failed: Invalid username or password (org.apache.kafka.clients.NetworkClient)

Zookeeper日志 #

[2024-08-01 11:25:16,533] INFO Successfully authenticated client: authenticationID=admin;  authorizationID=admin. (org.apache.zookeeper.server.auth.SaslServerCallbackHandler)
[2024-08-01 11:25:16,542] INFO Setting authorizedID: admin (org.apache.zookeeper.server.auth.SaslServerCallbackHandler)
[2024-08-01 11:25:16,543] INFO adding SASL authorization for authorizationID: admin (org.apache.zookeeper.server.ZooKeeperServer)

相关配置信息 #

Zookeeper启动与配置 #

• 启动命令：

  C:\kafka\bin\windows\zookeeper-server-start.bat C:\kafka\config\zookeeper.properties
  

• zookeeper-server-start.bat内容：

  IF [%1] EQU [] (
      echo USAGE: %0 zookeeper.properties
      EXIT /B 1
  )
  
  SetLocal
  IF ["%KAFKA_HEAP_OPTS%"] EQU [""] (
      set KAFKA_HEAP_OPTS=-Xmx512M -Xms512M
  )
  IF ["%KAFKA_OPTS%"] EQU [""] (
      set KAFKA_OPTS=-Djava.security.auth.login.config=C:\kafka\config\zookeeper_jaas.conf
  )
  
  "%~dp0kafka-run-class.bat" org.apache.zookeeper.server.quorum.QuorumPeerMain %*
  EndLocal
  

• zookeeper.properties内容：

  clientPort=2181
  maxClientCnxns=0
  admin.enableServer=false
  

• zookeeper_jaas.conf内容：

  Server {
      org.apache.zookeeper.server.auth.DigestLoginModule required
      user_admin="admin";
  };
  

Kafka Broker启动与配置 #

• 启动命令：

  C:\kafka\bin\windows\kafka-server-start.bat C:\kafka\config\server.properties --override ssl.keystore.location=C:\kafka\server.keystore.jks --override ssl.truststore.location=C:\kafka\server.truststore.jks --override ssl.keystore.password=12345 --override ssl.key.password=12345 --override ssl.truststore.password=12345
  

• kafka-server-start.bat内容：

  SetLocal
  IF ["%KAFKA_LOG4J_OPTS%"] EQU [""] (
      set KAFKA_LOG4J_OPTS=-Dlog4j.configuration=file:%~dp0../../config/log4j.properties
  )
  IF ["%KAFKA_OPTS%"] EQU [""] (
      set KAFKA_OPTS=-Djava.security.auth.login.config=C:\kafka\config\kafka_server_jaas.conf
  )
  "%~dp0kafka-run-class.bat" kafka.Kafka %*
  EndLocal
  

• server.properties内容：

  broker.id=0
  listeners=SASL_SSL://localhost:9093,SASL_PLAINTEXT://localhost:9092
  ssl.truststore.type=JKS
  ssl.keystore.type=pkcs12
  ssl.client.auth=required
  ssl.endpoint.identification.algorithm=
  inter.broker.listener.name=SASL_SSL
  listener.security.protocol.map=SASL_SSL:SASL_SSL,CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL
  sasl.enabled.mechanisms=PLAIN
  sasl.mechanism.controller.protocol=PLAIN
  sasl.mechanism.inter.broker.protocol=PLAIN
  authorizer.class.name=kafka.security.authorizer.AclAuthorizer
  allow.everyone.if.no.acl.found=false
  super.users=User:admin
  zookeeper.connection.timeout.ms=18000
  zookeeper.connect=localhost:2181
  

• kafka_server_jaas.conf内容：

  KafkaServer {
     org.apache.kafka.common.security.plain.PlainLoginModule required
     username="kafkabroker"
     password="kafkabroker-secret";
  };
  
  Client {
      org.apache.zookeeper.server.auth.DigestLoginModule required
      username="admin"
      password="admin";
  };
  

问题分析与修复步骤 #

核心问题是Broker间（含Controller与Broker）SASL认证失败，以及握手阶段请求异常，具体修复如下：

1. 补全Broker认证用户配置 #

PLAIN机制下，kafka_server_jaas.conf的KafkaServer段需显式定义允许连接的用户，否则Broker无法验证请求方的用户名密码。修改该段：

KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="kafkabroker"
   password="kafkabroker-secret"
   user_kafkabroker="kafkabroker-secret"
   user_admin="admin";
};

• user_kafkabroker：匹配Broker自身用户名，用于Broker间通信认证
• user_admin：匹配超级用户配置，确保管理员权限的连接能通过认证

2. 统一超级用户范围 #

将Broker自身用户名加入超级用户列表，避免Controller连接时权限不足：
修改server.properties：

super.users=User:admin,User:kafkabroker

3. 对齐SSL证书格式配置 #

启动命令中ssl.keystore.type设为pkcs12，但server.keystore.jks是JKS格式，需二选一调整：

• 若保留JKS证书，修改server.properties：ssl.keystore.type=JKS
• 若使用PKCS12格式，将证书文件转为.p12并更新启动命令中的路径

4. 重启服务 #

修改配置后，按顺序停止并重启服务：

# 停止服务
C:\kafka\bin\windows\zookeeper-server-stop.bat
C:\kafka\bin\windows\kafka-server-stop.bat

# 启动服务
C:\kafka\bin\windows\zookeeper-server-start.bat C:\kafka\config\zookeeper.properties
C:\kafka\bin\windows\kafka-server-start.bat C:\kafka\config\server.properties --override ssl.keystore.location=C:\kafka\server.keystore.jks --override ssl.truststore.location=C:\kafka\server.truststore.jks --override ssl.keystore.password=12345 --override ssl.key.password=12345 --override ssl.truststore.password=12345

内容的提问来源于stack exchange，提问作者ganvin