Kubernetes集群部署Kafka Pod失败：连接后断开问题求助

我尝试在Kubernetes集群中部署Kafka Pod，但Pod无法正常运行——创建后仅短暂连接就断开，调整多种配置仍未解决。以下是我的Zookeeper和Kafka的Deployment及Service配置：

Zookeeper Deployment #

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: zookeeper
  name: zookeeper-1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zookeeper
  template:
    metadata:
      labels:
        app: zookeeper
        name: zookeeper
    spec:
      containers:
        - name: zookeeper
          image: wurstmeister/zookeeper
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 2181
          env:
            - name: "ZOOKEEPER_CLIENT_PORT"
              value: "2181"
            - name: "ZOOKEEPER_TICK_TIME"
              value: "2000"

Zookeeper Service #

apiVersion: v1
kind: Service
metadata:
  name: zookeeper
spec:
  ports:
    - name: http
      port: 2181
  selector:
    app: zookeeper

Kafka Deployment #

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: kafka
  name: kafka-1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
        name: kafka
    spec:
      containers:
        - name: kafka
          image: wurstmeister/kafka
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 9092
          env:
            - name: KAFKA_BROKER_ID
              value: "1"
            - name: KAFKA_PORT
              value: "9092"  
            - name: KAFKA_ZOOKEEPER_CONNECT
              value: 10.102.191.88:2181
            - name: KAFKA_ADVERTISED_LISTENERS
              value: PLAINTEXT://kafka:9092
            - name: KAFKA_LISTENERS
              value: PLAINTEXT://0.0.0.0:9092
            - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
              value: PLAINTEXT:PLAINTEXT
            - name: KAFKA_INTER_BROKER_LISTENER_NAME
              value: PLAINTEXT
            - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
              value: "1"

Kafka Service #

apiVersion: v1
kind: Service
metadata:
  name: kafka
spec:
  ports:
    - name: http
      port: 9092
  selector:
    app: kafka

问题排查关键点 #

1. 硬编码Zookeeper IP不可靠：Kafka配置中的KAFKA_ZOOKEEPER_CONNECT使用了固定IP10.102.191.88:2181，Kubernetes中Service的IP是动态分配的，重启或重建Service后IP会变化，导致Kafka无法稳定连接Zookeeper。应改用Service域名zookeeper:2181，通过K8s DNS自动解析。
2. 冗余端口配置可能冲突：KAFKA_PORT属于冗余配置，wurstmeister/kafka镜像中KAFKA_LISTENERS已经明确指定了监听端口，保留该配置可能引发端口冲突。
3. Service端口命名不规范：Zookeeper和Kafka的Service端口命名为http，但两者均非HTTP服务，建议改为client或plaintext，避免配置混淆。

修复后的核心配置调整 #

修改Kafka Deployment中的环境变量，替换Zookeeper连接地址并移除冗余端口配置：

env:
  - name: KAFKA_BROKER_ID
    value: "1"
  # 移除冗余的KAFKA_PORT配置
  - name: KAFKA_ZOOKEEPER_CONNECT
    value: zookeeper:2181  # 使用Service域名替代固定IP
  - name: KAFKA_ADVERTISED_LISTENERS
    value: PLAINTEXT://kafka:9092
  - name: KAFKA_LISTENERS
    value: PLAINTEXT://0.0.0.0:9092
  - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
    value: PLAINTEXT:PLAINTEXT
  - name: KAFKA_INTER_BROKER_LISTENER_NAME
    value: PLAINTEXT
  - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
    value: "1"

同时可选调整Service的端口命名，提升可读性：

# Zookeeper Service端口修改
ports:
  - name: zookeeper-client
    port: 2181

# Kafka Service端口修改
ports:
  - name: kafka-plaintext
    port: 9092

验证步骤 #

1. 应用修改后的配置：kubectl apply -f <配置文件>
2. 查看Kafka Pod状态：kubectl get pods，确认Pod处于Running状态
3. 查看Pod日志排查是否还有连接错误：kubectl logs <kafka-pod-name>

内容的提问来源于stack exchange，提问作者Peleg Uzan