Kubernetes环境下Kafka与Zookeeper部署问题及外部通信咨询

Alright, let's walk through fixing your Kafka-Zookeeper connectivity issues and setting up the internal load balancer for external access step by step.

1. Fix Kafka-Zookeeper Communication Failures #

Looking at your configs, there are a few clear misconfigurations preventing Kafka from reaching Zookeeper:

a. Wrong Zookeeper DNS Name #

Your Kafka StatefulSet uses KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181", but you don't have a Service named zookeeper in your default namespace. Instead, your Zookeeper instances are exposed via services named zoo12, zoo21, and zoo31.

Since you're currently running a single Zookeeper replica (backed by the zoo31 service), update the env variable to:

- name: KAFKA_ZOOKEEPER_CONNECT
  value: "zoo31:2181"

If you plan to scale to a 3-node Zookeeper cluster later, list all three services: "zoo12:2181,zoo21:2181,zoo31:2181".

b. Invalid Headless Service for Kafka #

StatefulSets require a headless service to maintain stable network identities for pods, but your Kafka Service has an invalid clusterIP field (empty). Fix it by explicitly setting it to None:

apiVersion: v1
kind: Service
metadata:
  name: kafka
  namespace: default
spec:
  ports:
  - port: 9092
    name: plaintext
  clusterIP: None  # This marks it as a headless service
  selector:
    app: kafka

c. Mismatched Advertised Listeners #

Your KAFKA_ADVERTISED_LISTENERS uses the cp-quickstart namespace, but all your resources are in the default namespace. Correct the DNS name to match your setup:

- name: KAFKA_ADVERTISED_LISTENERS
  value: PLAINTEXT://broker-0.kafka.default.svc.cluster.local:9092

Also, the port 29092 in your original config isn't exposed by the Kafka container (you're using 9092), so stick with the container's port here.

2. Set Up Internal Load Balancer for External Access #

To expose Kafka to external clients via an internal load balancer, follow these steps:

a. Create an Internal LoadBalancer Service #

The exact annotations depend on your cloud provider, but here's a cloud-agnostic example with AWS-specific annotation included (adjust for GCP/Azure as needed):

apiVersion: v1
kind: Service
metadata:
  name: kafka-external
  namespace: default
  annotations:
    # AWS-specific annotation for internal LB
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    # GCP alternative: cloud.google.com/load-balancer-type: "Internal"
    # Azure alternative: service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  ports:
  - port: 9092
    targetPort: 9092
    name: plaintext-external
  selector:
    app: kafka

b. Update Kafka Listeners for External Access #

Configure Kafka to advertise both internal cluster listeners and external load balancer listeners. Modify the env variables in your Kafka StatefulSet:

env:
- name: KAFKA_ZOOKEEPER_CONNECT
  value: "zoo31:2181"
- name: KAFKA_LISTENERS
  value: PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:9093
- name: KAFKA_ADVERTISED_LISTENERS
  value: PLAINTEXT://broker-0.kafka.default.svc.cluster.local:9092,EXTERNAL://<EXTERNAL_LB_IP>:9092
- name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
  value: PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT

Don't forget to add the second container port to your Kafka container spec:

ports:
- containerPort: 9092
- containerPort: 9093

Replace <EXTERNAL_LB_IP> with the IP of your provisioned load balancer (get it via kubectl get service kafka-external).

c. Verify External Connectivity #

Once the load balancer is up, test external access with a Kafka client:

kafka-console-producer.sh --broker-list <EXTERNAL_LB_IP>:9092 --topic test-topic

3. Additional Checks #

• Zookeeper Cleanup: Your single-node Zookeeper deployment has env variables for ZOOKEEPER_SERVER_11 and ZOOKEEPER_SERVER_21 which aren't running. Remove these to avoid confusion, or deploy the other two Zookeeper nodes if you want a 3-node cluster.
• Pod Logs: If issues persist, check Kafka pod logs with kubectl logs broker-0 to see specific connection errors to Zookeeper.
• DNS Resolution: Exec into the Kafka pod and run nslookup zoo31 to confirm the Zookeeper service is resolvable via DNS.

内容的提问来源于stack exchange，提问作者Raju