启用SASL_PLAINTEXT后Kafka消息生产失败的问题排查求助

Alright, let's figure out why you're hitting that ClusterAuthorizationException even though your topic ACLs look correct. Here's the breakdown:

Root Cause Analysis #

The critical error here is Cluster authorization failed — this isn't about the topic-level permissions you've already configured, but missing cluster-level permissions for your producer user.

Here's the full context:

• When a Kafka producer starts up, it first needs to fetch cluster metadata (like broker addresses, topic partition details, etc.) to know where to send messages.
• Your error stack references TransactionManager.maybeAddPartition — this occurs when the producer tries to register topic partitions for message delivery, which relies on that cluster metadata lookup. Without cluster-level permissions to access this metadata, Kafka blocks the request with a cluster authorization error.

Additionally, your existing topic ACLs for the producer user have an empty host field (host= ). While this might work for localhost requests, it's unnecessarily restrictive and could cause issues if you ever run the producer from a different host.

Solutions #

Let's fix this step by step:

1. Add Cluster-Level DESCRIBE Permission #

Grant the producer user permission to fetch cluster metadata (required for producer initialization):

$KAFKA_HOME/kafka-acls.sh --bootstrap-server localhost:9093 --command-config client-properties/adminclient.properties --cluster --allow-principal User:producer --operation DESCRIBE --add

2. Fix Topic ACL Host Restriction #

Replace the restrictive empty host value with a wildcard (*) to allow the producer to connect from any host:

# First remove the old ACLs with empty host
$KAFKA_HOME/kafka-acls.sh --bootstrap-server localhost:9093 --command-config client-properties/adminclient.properties --topic first-topic --allow-principal User:producer --operation CREATE --host "" --remove
$KAFKA_HOME/kafka-acls.sh --bootstrap-server localhost:9093 --command-config client-properties/adminclient.properties --topic first-topic --allow-principal User:producer --operation DESCRIBE --host "" --remove
$KAFKA_HOME/kafka-acls.sh --bootstrap-server localhost:9093 --command-config client-properties/adminclient.properties --topic first-topic --allow-principal User:producer --operation WRITE --host "" --remove

# Add new ACLs with host=*
$KAFKA_HOME/kafka-acls.sh --bootstrap-server localhost:9093 --command-config client-properties/adminclient.properties --topic first-topic --allow-principal User:producer --producer --host "*" --add

3. Verify the Fix #

After applying these changes, run your producer command again:

kafka-console-producer --broker-list localhost:9093 --producer.config client-properties/producer.properties --topic first-topic

You should now be able to send messages without authorization errors.

Additional Checks #

• Double-check your producer.properties — your current config looks correct, but ensure there are no typos in the SASL JAAS username/password.
• Confirm your Kafka brokers have authorizer.class.name=kafka.security.authorizer.AclAuthorizer set in their configuration (this is required for ACLs to enforce properly).

内容的提问来源于stack exchange，提问作者Tom