AKS集群中MongoDB持续崩溃报错ECONNREFUSED求助
MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017 刚接触K8s,在Azure Kubernetes Service(AKS)上部署依赖MongoDB的微服务,AKS集群已创建,微服务运行正常,但MongoDB部署持续崩溃。日志报错:MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017
相关配置文件
mongo.yml
kind: Deployment metadata: name: mongo-deployment labels: app: mongodb spec: replicas: 1 selector: matchLabels: app: mongodb template: metadata: labels: app: mongodb spec: containers: - name: mongodb image: mongo ports: - containerPort: 27017 resources: requests: memory: "64Mi" cpu: "250m" limits: memory: "128Mi" cpu: "500m" env: - name: MONGO_INITDB_ROOT_USERNAME valueFrom: secretKeyRef: name: mongo-secret key: mongo-root-username - name: MONGO_INITDB_ROOT_PASSWORD valueFrom: secretKeyRef: name: mongo-secret key: mongo-root-password --- apiVersion: v1 kind: Service metadata: name: mongo-service spec: selector: app: mongodb ports: - protocol: TCP port: 27017 targetPort: 27017
mongo-configmap.yml
apiVersion: v1 kind: ConfigMap metadata: name: mongo-configmap data: connection_string: mongodb://username:password@mongo-service:27017
mongo-secret.yml
kind: Secret metadata: name: mongo-secret type: Opaque data: mongo-root-username: dXNlcm5hbWU= mongo-root-password: cGFzc3dvcmQ=
kubectl describe pod 输出
Namespace: default Priority: 0 Service Account: default Node: aks-nodepool1-85690495-vmss000000/10.224.0.4 Start Time: Tue, 25 Apr 2023 11:34:41 +0300 Labels: app=mongodb pod-template-hash=5d456c788f Annotations: <none> Status: Running IP: 10.244.0.37 IPs: IP: 10.244.0.37 Controlled By: ReplicaSet/mongo-deployment-5d456c788f Containers: mongodb: Container ID: containerd://6862b0a5291f7a4a6d1a7f81ba5b5fe2286f04a7d8aa7817cad2b48142190352 Image: mongo Image ID: docker.io/library/mongo@sha256:9c8a0a019671ed7d402768d4df6dddcc898828e21e9f7b90a34b55fe8ca676ac Port: 27017/TCP Host Port: 0/TCP State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: OOMKilled Exit Code: 1 Started: Tue, 25 Apr 2023 12:03:17 +0300 Finished: Tue, 25 Apr 2023 12:03:21 +0300 Ready: False Restart Count: 10 Limits: cpu: 500m memory: 128Mi Requests: cpu: 250m memory: 64Mi Environment: MONGO_INITDB_ROOT_USERNAME: <set to the key 'mongo-root-username' in secret 'mongo-secret'> Optional: false MONGO_INITDB_ROOT_PASSWORD: <set to the key 'mongo-root-password' in secret 'mongo-secret'> Optional: false Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-l6l84 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-l6l84: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: Burstable Node-Selectors: <none> Tolerations: node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 30m default-scheduler Successfully assigned default/mongo-deployment-5d456c788f-587r6 to aks-nodepool1-85690495-vmss000000 Normal Pulled 30m kubelet Successfully pulled image "mongo" in 735.583384ms (735.618884ms including waiting) Normal Pulled 30m kubelet Successfully pulled image "mongo" in 788.963634ms (788.974334ms including waiting) Normal Pulled 30m kubelet Successfully pulled image "mongo" in 725.867064ms (725.879864ms including waiting) Normal Created 29m (x4 over 30m) kubelet Created container mongodb Normal Started 29m (x4 over 30m) kubelet Started container mongodb Normal Pulled 29m kubelet Successfully pulled image "mongo" in 783.028263ms (783.042864ms including waiting) Normal Pulling 27m (x5 over 30m) kubelet Pulling image "mongo" Normal Pulled 27m kubelet Successfully pulled image "mongo" in 738.599919ms (738.611919ms including waiting) Warning BackOff 5m57s (x107 over 30m) kubelet Back-off restarting failed container
已尝试删除并重建Pod,问题仍存在,请求解决办法。
从kubectl describe pod输出可知,MongoDB容器终止原因是OOMKilled(内存不足被系统杀死),这是核心问题,结合配置细节给出以下解决步骤:
调整MongoDB内存资源限制
默认MongoDB镜像启动所需内存超过当前配置的128Mi限制,建议提升内存配额:
修改mongo.yml中resources部分:resources: requests: memory: "256Mi" cpu: "250m" limits: memory: "512Mi" cpu: "500m"修正微服务连接目标
日志中的127.0.0.1:27017连接错误,说明微服务可能错误地指向了本地而非MongoDB服务。确保微服务使用ConfigMap中的mongo-service:27017作为连接地址,同时确认ConfigMap中的用户名密码与Secret一致(当前Secret值为username和password,匹配ConfigMap配置)。添加持久化存储(可选但推荐)
当前部署未挂载持久卷,MongoDB每次重启会丢失数据,也可能因无存储目录导致启动异常。添加PersistentVolumeClaim:
在mongo.yml的containers节点下添加volumeMounts,同时在Pod的spec下添加volumes:containers: - name: mongodb # 其他原有配置 volumeMounts: - name: mongo-storage mountPath: /data/db volumes: - name: mongo-storage persistentVolumeClaim: claimName: mongo-pvc同时创建
mongo-pvc.yml:apiVersion: v1 kind: PersistentVolumeClaim metadata: name: mongo-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi应用配置更新
执行命令刷新部署:kubectl apply -f mongo.yml # 若添加了PVC,执行: kubectl apply -f mongo-pvc.yml验证启动状态
查看MongoDB容器详细日志确认是否启动成功:kubectl logs <mongo-pod-name>
内容的提问来源于stack exchange,提问作者Andrei Popescu

