AKS环境下Kubernetes API服务无法跨Pod访问问题求助

问题现象 #

• 在API Pod内访问localhost:4000可正常返回响应
• 访问API Service的名称web-api-service:8086或ClusterIP 10.0.171.39:8086均连接失败
• 从API Pod访问Web-UI Service web-ui-service:8084正常
• 尝试过ClusterIP、NodePort等多种Service类型，问题仍未解决

相关配置文件 #

API Deployment配置 #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mind-api
  namespace: mind-app
spec:
  selector:
    matchLabels:
      app: mind-api
  replicas: 1
  template:
    metadata:
      labels:
        app: mind-api
    spec:
      containers:
      - name: mind-api
        image: multiscale/mind-airflow:1.1.3
        imagePullPolicy: IfNotPresent
        ports:
          - containerPort: 4000
        env:
        - name: DB_HOST
          value: #####
        - name: DB_NAME
          value: #####
        - name: DB_USER
          value: #####
        - name: DB_PORT
          value: "27017"
        - name: SOURCE_MOUNTED_DRIVE
          value: "/mind-data"
        - name: MIND_HOME
          value: "/mind-data"
        volumeMounts:
            - name: azure-mind-vol
              mountPath: /mind-data
      volumes:
        - name: azure-mind-vol
          persistentVolumeClaim:
            claimName: mind-pvc
      imagePullSecrets:
        - name: regcred

API Service配置 #

apiVersion: v1
kind: Service
metadata:
  name: web-api-service
  namespace: mind-app
  labels:
    app: mind-api
spec:
  selector:
    app: mind-api
  ports:
    - protocol: "TCP"
      port: 8086
      targetPort: 4000
  type: LoadBalancer

集群Service列表 #

root@mindairflow:~/mind-api# kubectl get svc
NAME              TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)           AGE
mongo             LoadBalancer   10.0.176.106   ######   27017:31062/TCP   15d
web-api-service   LoadBalancer   10.0.171.39    #####    8086:30571/TCP    41h
web-ui-service    LoadBalancer   10.0.142.99    ######    8084:30909/TCP    16d

Pod内测试结果 #

访问API Service失败

/api # curl -i web-api-service:8086
curl: (7) Failed to connect to web-api-service port 8086 after 5 ms: Couldn't connect to server
/api # curl -i 10.0.171.39:8086
curl: (7) Failed to connect to 10.0.171.39 port 8086 after 0 ms: Couldn't connect to server

访问localhost:4000正常

/api # curl -i localhost:4000
HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, PATCH, DELETE
Access-Control-Allow-Headers: Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers,X-Access-Token,XKey,Authorization
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Type: text/html; charset=utf-8
Content-Length: 17
ETag: W/"11-uHyBxMeNlIJAQfw7PJuBJSmOJ9E"
Date: Sun, 13 Aug 2023 06:40:15 GMT
Connection: keep-alive
Keep-Alive: timeout=5

Server is running

访问Web-UI Service正常

/api # curl web-ui-service:8084
<!doctype html>
<html lang="en">
<head>
<!-- <link href="https://fonts.googleapis.com/icon?family=Material+Icons|Material+Icons+Outlined" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Lato:300,400,700|Roboto:100,300,400,500,700" rel="stylesheet"> -->

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  <meta http-equiv="Content Type" content="text/html; charset=UTF-8" />


  <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
  <meta http-equiv="Pragma" content="no-cache">
  <meta http-equiv="Expires" content="0">

  <title>MIND Web</title>
  <base href="/">

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="icon" type="image/x-icon" href="favicon.png">
</head>
<body>

  <app-root></app-root>
  <script id="xo-config" type="text/javascript"></script>
<script type="text/javascript" src="runtime.js"></script><script type="text/javascript" src="polyfills.js"></script><script type="text/javascript" src="styles.js"></script><script type="text/javascript" src="scripts.js"></script><script type="text/javascript" src="vendor.js"></script><script type="text/javascript" src="main.js"></script></body>
</html>

Dockerfile配置 #

FROM node:16.15.1-alpine3.16

RUN apk add git
WORKDIR /
COPY . /api
WORKDIR /api
RUN npm install --legacy-peer-deps
RUN apk add vim
RUN npm install -g forever

# 环境变量
ENV INTERNAL_IP="localhost" \
    PUBLIC_IP="http://localhost:4000" \
    DB_HOST="localhost" \
    DB_PORT=27017 \
    DB_NAME="mongo" \
    DB_USER="<用户名>" \
    DB_PASSWORD="" \
    AUTH_DB="<数据库名>" \
    SERVER_PORT=4000 \
    SERVER_HOST="localhost" \
    SUPER_ADMIN_USER="<邮箱地址>" \
    MAX_LICENSE_COUNT=100 \
    SHARED_DIR_PATH="<文件夹路径>" \
    PLATFORM_DATA_FOLDER_NAME="<文件夹名>" \
    HASH_SECRET_STRING="<密钥名>"

EXPOSE 4000
RUN chmod 777 -R /api
RUN chmod +x ./entrypoint.sh
#RUN chmod 755 ./entrypoint.sh
ENTRYPOINT [ "./entrypoint.sh" ]

排查与解决方案 #

核心问题定位 #

API服务在容器内仅绑定localhost地址，导致Kubernetes Service无法将集群网络的流量转发到容器。Kubernetes Service通过Pod的集群IP访问容器端口，而服务监听localhost时，只能接收容器内部的请求，无法响应来自集群其他节点或Service的请求。

解决步骤 #

1. 修改Deployment的环境变量：在Deployment的env字段中添加或覆盖SERVER_HOST为0.0.0.0，让服务监听容器的所有网卡地址：

   env:
   # 保留原有其他环境变量
   - name: SERVER_HOST
     value: "0.0.0.0"
   

2. 重新部署Deployment：

   kubectl apply -f <你的Deployment配置文件路径> -n mind-app
   

3. 验证连通性：
   • 等待Pod重启完成后，进入Pod再次测试访问web-api-service:8086
   • 也可以从集群内其他Pod或节点发起测试

额外验证步骤 #

• 检查Pod状态：kubectl get pods -n mind-app，确保Pod处于Running状态且就绪
• 检查Service端点：kubectl get endpoints web-api-service -n mind-app，确认列表中包含对应Pod的IP和端口
• 查看Pod日志：kubectl logs <目标Pod名称> -n mind-app，确认服务启动时监听地址为0.0.0.0:4000

内容的提问来源于stack exchange，提问作者Vivek Kumar Sinha