如何解决Kubernetes环境下Kafka与ZooKeeper的连接问题？

问题描述 #

部署时遇到错误：

Unable to canonicalize address zookeeper/:2181 because it's not resolvable

使用的YAML配置如下：

apiVersion: v1
kind: Namespace
metadata:
  name: kafka
---
apiVersion: v1
kind: Service
metadata:
  name: zookeeper-service
  namespace: kafka
spec:
  selector:
    app: zookeeper
  ports:
    - port: 2181
      targetPort: 2181
      name: client
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: zookeeper
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zookeeper
  template:
    metadata:
      labels:
        app: zookeeper
    spec:
      containers:
        - name: zookeeper
          image: confluentinc/cp-zookeeper:latest
          ports:
            - containerPort: 2181
          env:
            - name: ZOOKEEPER_CLIENT_PORT
              value: "2181"
---
apiVersion: v1
kind: Service
metadata:
  name: kafka-service
  namespace: kafka
spec:
  selector:
    app: kafka
  ports:
    - port: 9092
      targetPort: 9092
      name: broker
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
        - name: kafka
          image: confluentinc/cp-kafka:latest
          ports:
            - containerPort: 9092
          env:
            - name: KAFKA_BROKER_ID
              value: "1"
            - name: KAFKA_ZOOKEEPER_CONNECT
              value: "zookeeper-service:2181"
            - name: KAFKA_ADVERTISED_LISTENERS
              value: "PLAINTEXT://kafka-service:9092"
            - name: KAFKA_LISTENERS
              value: "PLAINTEXT://0.0.0.0:9092"
            - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
              value: "1"

Kafka Pod日志报错：

ERROR Unable to resolve address: zookeeper-service/:2181
(org.apache.zookeeper.client.StaticHostProvider) 2025-04-06 10:06:49
java.net.UnknownHostException: zookeeper-service: Name or service not
known 2025-04-06 10:06:49 at
java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
2025-04-06 10:06:49 at
java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(Unknown
Source) 2025-04-06 10:06:49 at
java.base/java.net.InetAddress.getAddressesFromNameService(Unknown
Source) 2025-04-06 10:06:49 at
java.base/java.net.InetAddress$NameServiceAddresses.get(Unknown
Source) 2025-04-06 10:06:49 at
java.base/java.net.InetAddress.getAllByName0(Unknown Source)
2025-04-06 10:06:49 at
java.base/java.net.InetAddress.getAllByName(Unknown Source) 2025-04-06
10:06:49 at java.base/java.net.InetAddress.getAllByName(Unknown
Source) 2025-04-06 10:06:49 at
org.apache.zookeeper.client.StaticHostProvider$1.getAllByName(StaticHostProvider.java:88)
2025-04-06 10:06:49 at
org.apache.zookeeper.client.StaticHostProvider.resolve(StaticHostProvider.java:141)
2025-04-06 10:06:49 at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:368)
2025-04-06 10:06:49 at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1204)
2025-04-06 10:06:49 [2025-04-06 04:06:49,752] WARN Session 0x0 for
server zookeeper-service/:2181, Closing socket connection.
Attempting reconnect except it is a SessionExpiredException.
(org.apache.zookeeper.ClientCnxn)

解决方案 #

1. 验证DNS解析与服务状态 #

• 检查ZooKeeper服务是否正常存在：

  kubectl get svc -n kafka
  

  确认zookeeper-service的状态为ClusterIP且有对应的端口映射。
• 进入Kafka Pod测试DNS解析：

  kubectl exec -it $(kubectl get pods -n kafka -l app=kafka -o jsonpath='{.items[0].metadata.name}') -n kafka -- nslookup zookeeper-service
  

  如果解析失败，检查Kubernetes集群的DNS组件（如CoreDNS）是否正常运行：

  kubectl get pods -n kube-system -l k8s-app=kube-dns
  

2. 补充ZooKeeper必要环境变量 #

Confluent的ZooKeeper镜像需要完整的配置参数才能正常启动，之前的配置缺少关键变量，导致ZooKeeper服务未正确初始化。修改ZooKeeper Deployment的env部分：

env:
  - name: ZOOKEEPER_CLIENT_PORT
    value: "2181"
  - name: ZOOKEEPER_SERVER_ID
    value: "1"
  - name: ZOOKEEPER_TICK_TIME
    value: "2000"
  - name: ZOOKEEPER_INIT_LIMIT
    value: "5"
  - name: ZOOKEEPER_SYNC_LIMIT
    value: "2"
  - name: ZOOKEEPER_SERVERS
    value: "zookeeper-service:2888:3888"

3. 替换ZooKeeper Deployment为StatefulSet #

ZooKeeper是有状态应用，StatefulSet能提供稳定的网络标识和持久化存储，比Deployment更适合。替换原ZooKeeper Deployment为以下配置：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zookeeper
  namespace: kafka
spec:
  serviceName: zookeeper-service
  replicas: 1
  selector:
    matchLabels:
      app: zookeeper
  template:
    metadata:
      labels:
        app: zookeeper
    spec:
      containers:
        - name: zookeeper
          image: confluentinc/cp-zookeeper:latest
          ports:
            - containerPort: 2181
            - containerPort: 2888
            - containerPort: 3888
          env:
            - name: ZOOKEEPER_CLIENT_PORT
              value: "2181"
            - name: ZOOKEEPER_SERVER_ID
              value: "1"
            - name: ZOOKEEPER_TICK_TIME
              value: "2000"
            - name: ZOOKEEPER_INIT_LIMIT
              value: "5"
            - name: ZOOKEEPER_SYNC_LIMIT
              value: "2"
            - name: ZOOKEEPER_SERVERS
              value: "zookeeper-service:2888:3888"

4. 清理并重新部署 #

删除现有资源后重新应用配置，确保资源按顺序创建：

kubectl delete -f your-config.yaml
kubectl apply -f your-config.yaml

等待ZooKeeper Pod完全启动后，再检查Kafka Pod的连接状态。

内容的提问来源于stack exchange，提问作者hrdfgn