You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

同一份Kong部署YAML在GKE正常,MacOS本地MicroK8s探针失败

Kong在MacOS MicroK8s环境中探针超时问题排查

问题背景

相同的Kong Deployment YAML在GKE集群部署后运行正常,但在MacOS本地MicroK8s环境中,Pod状态显示Running,但就绪探针(Readiness Probe)和存活探针(Liveness Probe)频繁出现超时失败警告,报错信息为:

Readiness probe failed: Get "http://10.1.254.79:8100/status": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

部署YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: local-test-kong
  labels:
    app: local-test-kong
spec:
  replicas: 1
  selector:
    matchLabels:
      app: local-test-kong
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: local-test-kong
    spec:
      automountServiceAccountToken: false
      containers:
        - envFrom:
            - configMapRef:
                name: kong-env-vars
          image: kong:2.6
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /bin/sh
                  - -c
                  - /bin/sleep 15 && kong quit
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /status
              port: status
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          name: proxy
          ports:
            - containerPort: 8000
              name: proxy
              protocol: TCP
            - containerPort: 8100
              name: status
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /status
              port: status
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          resources: # ToDo
            limits:
              cpu: 256m
              memory: 256Mi
            requests:
              cpu: 256m
              memory: 256Mi
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /kong_prefix/
              name: kong-prefix-dir
            - mountPath: /tmp
              name: tmp-dir
            - mountPath: /kong_dbless/
              name: kong-custom-dbless-config-volume
      terminationGracePeriodSeconds: 30
      volumes:
        - name: kong-prefix-dir
        - name: tmp-dir
        - configMap:
            defaultMode: 0555
            name: kong-declarative
          name: kong-custom-dbless-config-volume

GKE环境Pod状态(无异常)

➜  kubectl get pods
NAME                               READY   STATUS    RESTARTS   AGE
local-test-kong-678598ffc6-ll9s8   1/1     Running   0          25m
➜  kubectl describe pod/local-test-kong-678598ffc6-ll9s8
Name:         local-test-kong-678598ffc6-ll9s8
Namespace:    local-test-kong
Priority:     0
Node:         gke-paas-cluster-prd-tf9-default-pool-e7cb502a-ggxl/10.128.64.95
Start Time:   Wed, 23 Nov 2022 00:12:56 +0800
Labels:       app=local-test-kong
              pod-template-hash=678598ffc6
Annotations:  kubectl.kubernetes.io/restartedAt: 2022-11-23T00:12:56+08:00
Status:       Running
IP:           10.128.96.104
IPs:
  IP:           10.128.96.104
Controlled By:  ReplicaSet/local-test-kong-678598ffc6
Containers:
  proxy:
    Container ID:   containerd://1bd392488cfe33dcc62f717b3b8831349e8cf573326add846c9c843c7bf15e2a
    Image:          kong:2.6
    Image ID:       docker.io/library/kong@sha256:62eb6d17133b007cbf5831b39197c669b8700c55283270395b876d1ecfd69a70
    Ports:          8000/TCP, 8100/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Running
      Started:      Wed, 23 Nov 2022 00:12:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     256m
      memory:  256Mi
    Requests:
      cpu:      256m
      memory:   256Mi
    Liveness:   http-get http://:status/status delay=10s timeout=5s period=10s #success=1 #failure=3
    Readiness:  http-get http://:status/status delay=10s timeout=5s period=10s #success=1 #failure=3
    Environment Variables from:
      kong-env-vars  ConfigMap  Optional: false
    Environment:     <none>
    Mounts:
      /kong_dbless/ from kong-custom-dbless-config-volume (rw)
      /kong_prefix/ from kong-prefix-dir (rw)
      /tmp from tmp-dir (rw)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kong-prefix-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  tmp-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  kong-custom-dbless-config-volume:
    Type:        ConfigMap (a volume populated by a ConfigMap)
    Name:        kong-declarative
    Optional:    false
QoS Class:       Guaranteed
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  25m   default-scheduler  Successfully assigned local-test-kong/local-test-kong-678598ffc6-ll9s8 to gke-paas-cluster-prd-tf9-default-pool-e7cb502a-ggxl
  Normal  Pulled     25m   kubelet            Container image "kong:2.6" already present on machine
  Normal  Created    25m   kubelet            Created container proxy
  Normal  Started    25m   kubelet            Started container proxy

MacOS MicroK8s环境Pod状态(探针失败)

➜  kubectl get pods
NAME                               READY   STATUS    RESTARTS   AGE
local-test-kong-54cfc585cb-7grj8   1/1     Running   0          86s
➜  kubectl describe pod/local-test-kong-54cfc585cb-7grj8
Name:         local-test-kong-54cfc585cb-7grj8
Namespace:    local-test-kong
Priority:     0
Node:         microk8s-vm/192.168.64.5
Start Time:   Wed, 23 Nov 2022 00:39:33 +0800
Labels:       app=local-test-kong
              pod-template-hash=54cfc585cb
Annotations:  cni.projectcalico.org/podIP: 10.1.254.79/32
              cni.projectcalico.org/podIPs: 10.1.254.79/32
              kubectl.kubernetes.io/restartedAt: 2022-11-23T00:39:33+08:00
Status:       Running
IP:           10.1.254.79
IPs:
  IP:           10.1.254.79
Controlled By:  ReplicaSet/local-test-kong-54cfc585cb
Containers:
  proxy:
    Container ID:   containerd://d60d09ca8b77ee59c80ea060dcb651c3e346c3a5f0147b0d061790c52193d93d
    Image:          kong:2.6
    Image ID:       docker.io/library/kong@sha256:62eb6d17133b007cbf5831b39197c669b8700c55283270395b876d1ecfd69a70
    Ports:          8000/TCP, 8100/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Running
      Started:      Wed, 23 Nov 2022 00:39:37 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     256m
      memory:  256Mi
    Requests:
      cpu:      256m
      memory:   256Mi
    Liveness:   http-get http://:status/status delay=10s timeout=5s period=10s #success=1 #failure=3
    Readiness:  http-get http://:status/status delay=10s timeout=5s period=10s #success=1 #failure=3
    Environment Variables from:
      kong-env-vars  ConfigMap  Optional: false
    Environment:     <none>
    Mounts:
      /kong_dbless/ from kong-custom-dbless-config-volume (rw)
      /kong_prefix/ from kong-prefix-dir (rw)
      /tmp from tmp-dir (rw)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kong-prefix-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  tmp-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  kong-custom-dbless-config-volume:
    Type:        ConfigMap (a volume populated by a ConfigMap)
    Name:        kong-declarative
    Optional:    false
QoS Class:       Guaranteed
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  92s   default-scheduler  Successfully assigned local-test-kong/local-test-kong-54cfc585cb-7grj8 to microk8s-vm
  Normal   Pulled     90s   kubelet            Container image "kong:2.6" already present on machine
  Normal   Created    90s   kubelet            Created container proxy
  Normal   Started    89s   kubelet            Started container proxy
  Warning  Unhealthy  68s   kubelet            Readiness probe failed: Get "http://10.1.254.79:8100/status": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy  68s   kubelet            Liveness probe failed: Get "http://10.1.254.79:8100/status": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

排查方向及解决方案

1. 调整探针启动延迟和超时时间

本地MicroK8s的VM资源通常比GKE节点有限,Kong加载配置和启动服务的时间更长,当前initialDelaySeconds:10可能不足以让状态端口8100完全就绪。

  • 修改Deployment中的探针配置:
    livenessProbe:
      initialDelaySeconds: 30
      timeoutSeconds: 10
    readinessProbe:
      initialDelaySeconds: 30
      timeoutSeconds: 10
    
  • 重启Deployment:
    kubectl rollout restart deployment/local-test-kong -n local-test-kong
    

2. 检查Kong状态端口绑定配置

确认kong-env-vars ConfigMap中,Kong的状态监听地址是否绑定到0.0.0.0:8100,如果仅绑定到localhost,kubelet无法从外部访问该端口。

  • 编辑ConfigMap:
    kubectl edit configmap/kong-env-vars -n local-test-kong
    
  • 添加或修改变量:
    data:
      KONG_STATUS_LISTEN: "0.0.0.0:8100"
    
  • 重启Deployment生效。

3. 排查MicroK8s网络插件限制

MicroK8s默认使用Calico作为CNI插件,可能存在网络策略阻止kubelet访问Pod端口。

  • 临时删除命名空间下所有网络策略测试:
    kubectl delete networkpolicy --all -n local-test-kong
    
  • 如果恢复正常,需要调整Calico网络策略,允许kubelet所在节点访问Pod的8100端口。

4. 优化EmptyDir卷性能

Deployment中kong-prefix-dir和tmp-dir使用默认EmptyDir(磁盘介质),本地VM磁盘性能可能较差,导致Kong启动延迟。

  • 修改为内存介质的EmptyDir:
    volumes:
      - name: kong-prefix-dir
        emptyDir:
          medium: Memory
      - name: tmp-dir
        emptyDir:
          medium: Memory
    
  • 重启Deployment生效。

5. 测试Pod内部端口连通性

进入Pod内部测试状态端口是否正常监听:

kubectl exec -it local-test-kong-54cfc585cb-7grj8 -n local-test-kong -- curl http://localhost:8100/status

如果返回正常状态,说明问题出在外部网络访问;如果也超时,说明Kong本身未正常启动状态服务,需要检查日志:

kubectl logs local-test-kong-54cfc585cb-7grj8 -n local-test-kong

内容的提问来源于stack exchange,提问作者Rakib

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.08.11 12:10:27