Kubernetes上的Kafka集群：客户端无法获取元数据

Let’s walk through the most likely culprits for your metadata retrieval problem—Kubernetes adds some extra layers to Kafka networking, so small misconfigurations here are common.

1. Verify Your advertised.listeners Configuration #

The biggest red flag here is the {EXTERNAL-IP-ADDRESS} placeholder in your advertised.listeners setting. Let’s make sure this is correctly populated:

• If you’re using a LoadBalancer service for Kafka: Run kubectl get svc and check the EXTERNAL-IP column for your Kafka service. Replace {EXTERNAL-IP-ADDRESS} with this actual IP. Also confirm the service maps the external port (e.g., 9093) directly to the container’s 9093 port.
• If you’re using a NodePort service: Use the public IP of one of your Kubernetes nodes, and replace the port in advertised.listeners with the NodePort value (not 9093—you can find this in the PORT(S) column of kubectl get svc output).
• Double-check that the PLAINTEXT_HOST entry in advertised.listeners exactly matches what your external client will use to connect.

2. Validate Listener Protocol Mapping & Client Configuration #

• Your listener.security.protocol.map looks correct (PLAINTEXT_HOST:PLAINTEXT), but ensure your external client is using the right settings:
  • Bootstrap servers should point to the external address/port from advertised.listeners
  • Set security.protocol=PLAINTEXT in your client config (since that’s what PLAINTEXT_HOST uses)

3. Check Network Accessibility #

• Cluster Firewall/Security Groups: Confirm the port you’re using for external access (9093 or your NodePort) is open to your client’s IP range.
• Kubernetes NetworkPolicies: If you have NetworkPolicies enabled, make sure there’s a policy allowing ingress traffic to your Kafka service on the external listener port.
• Test Connectivity: From your client machine, run a simple port check like telnet {EXTERNAL-IP} 9093 or nc -zv {EXTERNAL-IP} 9093 to confirm the port is reachable.

4. Ensure ZooKeeper Connectivity #

Your Kafka brokers rely on ZooKeeper to store cluster metadata. If they can’t reach ZK, clients won’t get valid metadata either:

• In one of your Kafka pods, run kubectl exec -it <kafka-pod-name> -- nc -zv zk-cs.analytics 2181 (replace 2181 with your ZK port if different) to test connectivity.
• Check Kafka pod logs for ZK-related errors with kubectl logs <kafka-pod-name> | grep -i zookeeper.

5. Inspect Kafka Broker Logs #

Logs will often tell you exactly what’s wrong:

• Run kubectl logs <kafka-pod-name> and look for messages about:
  • Failed listener registration
  • Mismatches between listeners and advertised.listeners
  • Metadata storage errors in ZooKeeper

If you share specific log snippets or your Kubernetes service definition, we can narrow this down further!

内容的提问来源于stack exchange，提问作者j9dy