K8s前端mongo-express无法连接后端MongoDB服务求助

环境概述 #

搭建了包含前端mongo-express和后端MongoDB的K8s集群：

• 集群由1个master节点+2个worker节点组成
• master节点和私有worker节点处于私有子网，通过AWS NAT Gateway访问互联网
• 公有worker节点处于公有子网，通过Internet Gateway访问互联网
• 后端MongoDB应用运行正常，但前端mongo-express启动失败

报错信息 #

Welcome to mongo-express
------------------------


(node:8) [MONGODB DRIVER] Warning: Current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
Could not connect to database using connectionString: mongodb://mongousername:undefined@mongodb-service:27017/"
(node:8) UnhandledPromiseRejectionWarning: MongoNetworkError: failed to connect to server [mongodb-service:27017] on first connect [Error: getaddrinfo EAI_AGAIN mongodb-service
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26) {
  name: 'MongoNetworkError'
}]
    at Pool.<anonymous> (/node_modules/mongodb/lib/core/topologies/server.js:441:11)
    at Pool.emit (events.js:314:20)
    at /node_modules/mongodb/lib/core/connection/pool.js:564:14
    at /node_modules/mongodb/lib/core/connection/pool.js:1000:11
    at /node_modules/mongodb/lib/core/connection/connect.js:32:7
    at callback (/node_modules/mongodb/lib/core/connection/connect.js:300:5)
    at Socket.<anonymous> (/node_modules/mongodb/lib/core/connection/connect.js:330:7)
    at Object.onceWrapper (events.js:421:26)
    at Socket.emit (events.js:314:20)
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
(node:8) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:8) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

K8s配置文件 #

前端（mongo-express）配置 #

apiVersion: v1
kind: ConfigMap
metadata:
 name: mongodb-configmap
data:
 database_url: mongodb-service

---

apiVersion: apps/v1
kind: Deployment
metadata:
 name: mongo-express
 labels:
   app: mongo-express
spec:
 replicas: 1
 selector:
   matchLabels:
     app: mongo-express
 template:
   metadata:
     labels:
       app: mongo-express
   spec:
     affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: app
                operator: In
                values:
                - frontend
     containers:
     - name: mongo-express
       image: mongo-express
       ports:
       - containerPort: 8081
       env:
       - name: ME_CONFIG_MONGODB_ADMINUSERNAME
         valueFrom:
           secretKeyRef:
             name: mongodb-secret
             key: mongodb-root-username
       - name: ME_CONFIG_MONGODB_ADMINPASSWOR
         valueFrom:
           secretKeyRef:
             name: mongodb-secret
             key: mongodb-root-password
       - name: ME_CONFIG_MONGODB_SERVER
         valueFrom:
           configMapKeyRef:
             name: mongodb-configmap
             key: database_url
---

apiVersion: v1
kind: Service
metadata:
 name: mongo-express-service
spec:
 selector:
   app: mongo-express
 type: LoadBalancer
 ports:
   - protocol: TCP
     port: 8081
     targetPort: 8081
     nodePort: 30000

后端（MongoDB）配置 #

apiVersion: v1
kind: Secret
metadata:
 name: mongodb-secret
type: Opaque
data:
 mongodb-root-username: bW9uZ291c2VybmFtZQ==
 mongodb-root-password: bW9uZ29wYXNzd29yZA==

---

apiVersion: apps/v1

kind: Deployment # mongodb deployment
metadata:
 name: mongodb-deployment
 labels:
   app: mongodb
spec:
 replicas: 1
 selector:
   matchLabels:
     app: mongodb
 template:
   metadata:
     labels:
       app: mongodb
   spec: # mongodb deployment spec
     affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: app
                operator: In
                values:
                - backend
     containers:
       - name: mongodb-container
         image: mongo # mongo image name
         ports: # mongodb container ports
         - containerPort: 27017
         env: # mongodb container env variables
         - name: MONGO_INITDB_ROOT_USERNAME
           valueFrom: # mongodb env variable value from secret
             secretKeyRef:
               name: mongodb-secret
               key: mongodb-root-username
         - name: MONGO_INITDB_ROOT_PASSWORD
           valueFrom: # mongodb ev variable from secret
             secretKeyRef:
               name: mongodb-secret
               key: mongodb-root-password

---

apiVersion: v1
kind: Service
metadata:
 name: mongodb-service
 annotations:
    service.beta.kubernetes.io/load-balancer-internal: "true"
spec:
 selector:
   app: mongodb
 ports:
   - protocol: TCP
     port: 27017
     targetPort: 27017

补充排查结果 #

执行kubectl get svc输出：

NAME              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)     AGE
kubernetes        ClusterIP   10.96.0.1      <none>        443/TCP     3d7h
mongodb-service   ClusterIP   10.100.40.68   <none>        27017/TCP   2d15h

执行kubectl describe service mongodb-service输出：

Name:              mongodb-service
Namespace:         default
Labels:            <none>
Annotations:       service.beta.kubernetes.io/load-balancer-internal: true
Selector:          app=mongodb
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.100.40.68
IPs:               10.100.40.68
Port:              <unset>  27017/TCP
TargetPort:        27017/TCP
Endpoints:         10.44.0.1:27017
Session Affinity:  None
Events:            <none>

问题分析与修复步骤 #

问题1：环境变量名称拼写错误 #

从报错信息里的连接字符串mongodb://mongousername:undefined@mongodb-service:27017/可以看出，密码字段为undefined，这是因为前端Deployment中环境变量名称拼写错误：

• 错误名称：ME_CONFIG_MONGODB_ADMINPASSWOR
• 正确名称：ME_CONFIG_MONGODB_ADMINPASSWORD（少了最后一个字母d）

修复方法：
修改前端Deployment的env部分，将错误的环境变量名称替换为正确名称，然后重新应用配置：

kubectl apply -f <前端配置文件路径>

问题2：DNS解析失败 #

报错中的getaddrinfo EAI_AGAIN mongodb-service说明mongo-express容器无法解析MongoDB服务的DNS名称。结合部署架构：

• mongo-express被调度到公有子网的frontend节点
• MongoDB被调度到私有子网的backend节点

需要做以下验证和修复：

1. 验证CoreDNS运行状态：
   执行kubectl get pods -n kube-system | grep coredns，确认CoreDNS Pod处于Running状态。

2. 在mongo-express容器内测试连通性：
   进入mongo-express的Pod：

   kubectl exec -it <mongo-express-pod-name> -- /bin/sh
   

   执行以下命令测试：

   • 解析服务DNS：nslookup mongodb-service，如果无法解析，检查CoreDNS是否正常，或者Pod的DNS配置是否正确。
   • 直接访问ClusterIP：ping 10.100.40.68（mongodb-service的ClusterIP），如果能ping通但DNS解析失败，检查Pod的/etc/resolv.conf配置是否指向CoreDNS的ClusterIP（通常是10.96.0.10）。
   • 测试端口连通性：telnet 10.100.40.68 27017，如果无法连通，检查AWS子网的安全组规则，确保公有子网和私有子网之间允许TCP 27017端口的流量。

3. 检查节点间网络策略：
   如果集群中配置了NetworkPolicy，确认没有阻止mongo-express Pod访问MongoDB服务的规则。

内容的提问来源于stack exchange，提问作者George Udosen