Kubernetes中Kafka无法与ZooKeeper通信问题排查求助

当前状态 #

• my-k8s-zookeeper 部署成功（对应ZooKeeper Pod运行状态截图）
• ZooKeeper关联Service显示正常运行（对应Service状态截图）
• my-k8s-kafka 已重启3次，持续报错无法连接ZooKeeper Service，Service状态正常但通信失败（对应Kafka报错日志截图及补充截图）

提供的配置文件 #

kafka-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-k8s-kafka
spec:
replicas: 1
selector:
  matchLabels:
    component: my-kafka
template:
  metadata:
    labels:
      component: my-kafka
  spec:
    containers:
      - name: my-k8s-container-kafka
        image: confluentinc/cp-kafka:latest
        ports:
          - containerPort: 9092
        env:
          - name: KAFKA_ZOOKEEPER_CONNECT
            value: "my-k8s-zookeeper-service:2181"
          - name: KAFKA_ADVERTISED_LISTENERS
            value: "PLAINTEXT://my-k8s-kafka-service:29092,PLAINTEXT_HOST://my-k8s-kafka-service:9092"
          - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
            value: "PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT"
          - name: KAFKA_INTER_BROKER_LISTENER_NAME
            value: "PLAINTEXT"
          - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
            value: "1"
          - name: KAFKA_BROKER_ID
            value: "1"
          - name: KAFKA_MIN_INSYNC_REPLICAS
            value: "1"

kafka-deployment-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-k8s-kafka-service
spec:
  type: ClusterIP
  selector:
    component: my-k8s-kafka
  ports:
    - port: 9092
      targetPort: 9092

zookeeper-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-k8s-zookeeper
spec:
  replicas: 1
  selector:
    matchLabels:
      component: my-zookeeper
  template:
    metadata:
      labels:
        component: my-zookeeper
    spec:
      containers:
        - name: my-k8s-container-zookeeper
          image: confluentinc/cp-zookeeper:latest
          env:
            - name: ZOOKEEPER_CLIENT_PORT
              value: "2181"
            - name: ZOOKEEPER_TICK_TIME
              value: "2000"
          ports:
            - containerPort: 2181
          resources:
            limits:
              cpu: 500m
              memory: 1Gi

zookeeper-deployment-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-k8s-zookeeper-service
spec:
  type: ClusterIP
  selector:
    component: my-k8s-zookeeper
  ports:
    - port: 2181
      targetPort: 2181

问题根源及解决方法 #

核心问题：ZooKeeper Service的Selector与Pod标签不匹配 #

ZooKeeper Deployment中定义的Pod标签是component: my-zookeeper，但ZooKeeper Service的selector配置为component: my-k8s-zookeeper，两者完全不一致，导致Service无法关联到ZooKeeper Pod，Kafka自然无法通过Service访问到ZooKeeper。

修复步骤 #

1. 修改zookeeper-deployment-service.yaml中的selector，与Pod标签保持一致：

apiVersion: v1
kind: Service
metadata:
  name: my-k8s-zookeeper-service
spec:
  type: ClusterIP
  selector:
    component: my-zookeeper  # 修正为与Pod标签一致的值
  ports:
    - port: 2181
      targetPort: 2181

2. 重新应用修改后的Service配置：

kubectl apply -f zookeeper-deployment-service.yaml

3. 验证Service与Pod的关联状态：

kubectl describe service my-k8s-zookeeper-service

查看输出中的Endpoints字段，确认已关联到ZooKeeper Pod的IP和端口。

额外验证步骤 #

• 查看ZooKeeper Pod的就绪状态：

kubectl get pods -l component=my-zookeeper

确保Pod处于Running且READY状态为1/1。

• 进入Kafka Pod内部测试连通性：

kubectl exec -it $(kubectl get pods -l component=my-kafka -o name) -- nc -zv my-k8s-zookeeper-service 2181

如果连接成功，会显示Connection succeeded。

内容的提问来源于stack exchange，提问作者jeremy