Kafka Kraft集群配置错误：JAAS配置缺失KafkaServer相关条目

问题背景 #

• 维护Kafka Kraft集群（3控制器+3Broker，非混合模式），目标构建加密、支持ACL的安全集群
• 集群配置：节点用内部FQDN管理，Broker分配第三方CA签发的SAN公网FQDN；已完成TLS加密，外网测试收发数据正常
• 启用SASL_SCRAM-512认证后，启动Controller节点出现致命错误

错误日志 #

[2024-01-27 12:26:20,301] ERROR Encountered fatal fault: caught exception (org.apache.kafka.server.fault.ProcessTerminatingFaultHandler)
java.lang.IllegalArgumentException: Could not find a 'KafkaServer' or 'controller.KafkaServer' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
        at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:150)
        at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:103)
        at org.apache.kafka.common.security.JaasContext.loadServerContext(JaasContext.java:74)
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:143)
        at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:107)
        at kafka.network.Processor.<init>(SocketServer.scala:973)
        at kafka.network.Acceptor.newProcessor(SocketServer.scala:879)
        at kafka.network.Acceptor.$anonfun$addProcessors$1(SocketServer.scala:849)
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
        at kafka.network.Acceptor.addProcessors(SocketServer.scala:848)
        at kafka.network.DataPlaneAcceptor.configure(SocketServer.scala:523)
        at kafka.network.SocketServer.createDataPlaneAcceptorAndProcessors(SocketServer.scala:251)
        at kafka.network.SocketServer.$anonfun$new$29(SocketServer.scala:172)
        at kafka.network.SocketServer.$anonfun$new$29$adapted(SocketServer.scala:172)
        at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
        at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:933)
        at kafka.network.SocketServer.<init>(SocketServer.scala:172)
        at kafka.server.ControllerServer.startup(ControllerServer.scala:188)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:95)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:95)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95)
        at kafka.Kafka$.main(Kafka.scala:113)
        at kafka.Kafka.main(Kafka.scala)

原有正常TLS配置 #

controller.properties #

process.roles=controller
node.id=1
controller.quorum.voters=1@kafcon01-1.internal.local:9092,2@kafcon01-2.internal.local:9092,3@kafcon01-3.internal.local:9092
listeners=CONTROLLER://kafcon01-1.internal.local:9092
controller.listener.names=CONTROLLER
listener.security.protocol.map=CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT,CLIENT:SSL
ssl.keystore.type=PEM
ssl.keystore.location=/opt/kafka/ssl/fqdn_fullstack.pem
ssl.client.auth=requested
ssl.endpoint.identification.algorithm=

broker.properties #

process.roles=broker
node.id=4
controller.quorum.voters=1@kafcon01-1.internal.local:9092,2@kafcon01-2.internal.local:9092,3@kafcon01-3.internal.local:9092
listeners=BROKER://kafbrk01-4.internal.local:9092,CLIENT://kafbrk01-4.fqdn:9093
inter.broker.listener.name=BROKER
controller.listener.names=CONTROLLER
listener.security.protocol.map=CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT,CLIENT:SSL
ssl.keystore.type=PEM
ssl.keystore.location=/opt/kafka/ssl/fqdn_fullstack.pem
ssl.client.auth=requested
ssl.endpoint.identification.algorithm=

修改后的安全配置 #

listener.security.protocol.map=CONTROLLER:SASL_PLAINTEXT,BROKER:SASL_PLAINTEXT,CLIENT:SASL_SSL
sasl.enabled.mechanisms=SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
sasl.mechanism.controller.protocol=SCRAM-SHA-512
super.users=User:admin
authorizer.class.name=org.apache.kafka.common.security.scram.ScramLoginModule
listener.name.sasl_plaintext.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="admin" \
    password="admin-secret";
listener.name.sasl_ssl.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required;

执行的格式化命令 #

/opt/kafka/bin/kafka-storage.sh format --cluster-id <cluserid> --config /opt/kafka/config/kraft/controller.properties --add-scram "SCRAM-SHA-512=[name=admin,password=admin-secret"

解决方案 #

1. 修复JAAS配置缺失问题 #

错误核心是未指定JAAS配置文件，需补充以下步骤：

• 创建JAAS配置文件（如/opt/kafka/config/kafka_jaas.conf）：

  KafkaServer {
      org.apache.kafka.common.security.scram.ScramLoginModule required
      username="admin"
      password="admin-secret";
  };
  Controller {
      org.apache.kafka.common.security.scram.ScramLoginModule required
      username="admin"
      password="admin-secret";
  };
  

• 启动Controller和Broker时，通过JVM参数指定JAAS路径：

  export KAFKA_OPTS="-Djava.security.auth.login.config=/opt/kafka/config/kafka_jaas.conf"
  

  可将此配置写入kafka-server-start.sh脚本，避免每次启动手动添加。

2. 修正授权器配置错误 #

authorizer.class.name配置错误，ScramLoginModule是登录模块，不是授权器：

# 替换原有错误配置
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
allow.everyone.if.no.acl.found=false
super.users=User:admin

3. 修复格式化命令语法错误 #

原命令括号未闭合，正确格式：

/opt/kafka/bin/kafka-storage.sh format --cluster-id <clusterid> --config /opt/kafka/config/kraft/controller.properties --add-scram "SCRAM-SHA-512=[name=admin,password=admin-secret]"

4. 补充Controller Listener的JAAS配置 #

为CONTROLLER listener添加对应SASL配置：

listener.name.controller.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="admin" \
    password="admin-secret";

优化建议 #

1. 内部通信加密升级 #

将CONTROLLER、BROKER listener改为SASL_SSL，增强内部通信安全性：

listener.security.protocol.map=CONTROLLER:SASL_SSL,BROKER:SASL_SSL,CLIENT:SASL_SSL

复用现有TLS证书配置，实现加密+认证双重保障。

2. 用户管理最佳实践 #

• 避免配置文件明文存密码，后续用户管理用kafka-configs.sh工具：

  /opt/kafka/bin/kafka-configs.sh --bootstrap-server <broker-addr:9093> --command-config admin-client.properties --alter --add-config 'SCRAM-SHA-512=[password=admin-secret]' --entity-type users --entity-name admin
  

• 拆分权限：创建生产、只读等专用用户，避免用admin处理业务请求。

3. TLS配置增强 #

• 生产环境启用ssl.endpoint.identification.algorithm=HTTPS，防止中间人攻击
• 设置ssl.client.auth=required，强制客户端提供证书，启用双向认证

4. 启动脚本标准化 #

在kafka-server-start.sh中统一配置KAFKA_OPTS，包含JAAS路径、堆内存等参数，简化启动流程。

内容的提问来源于stack exchange，提问作者Scott C