Pod状态CrashLoopBackOff无法启动:initContainer正常主容器异常求助
排查Pod CrashLoopBackOff问题
问题背景
原本通过ConfigMap注入单个URL作为环境变量时,DaemonSet的Pod可正常启动。现在因存在多个URL,改为通过initContainer筛选有效URL并写入/tmp/ip.env,再在主容器中设置环境变量,但initContainer执行无报错,主容器却陷入CrashLoopBackOff状态。
原正常配置
apiVersion: apps/v1 kind: DaemonSet metadata: name: {{ .Chart.Name }} annotations: reloader.stakater.com/auto: "true" spec: selector: matchLabels: some-service: {{ .Chart.Name }} template: #required fields spec: hostNetwork: yes containers: - name: {{ .Chart.Name }} image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.image.pullPolicy }} env: - name: HOST_URL valueFrom: configMapKeyRef: name: some-config key: some-url
修改后的配置
apiVersion: apps/v1 kind: DaemonSet metadata: name: {{ .Chart.Name }} annotations: reloader.stakater.com/auto: "true" spec: selector: matchLabels: some-service: {{ .Chart.Name }} template: #required fields spec: hostNetwork: yes containers: - name: {{ .Chart.Name }} image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.image.pullPolicy }} command: - sh - -c - env HOST_URL=$(cat /tmp/ip.env) volumeMounts: - name: workdir mountPath: /tmp initContainers: - name: wait-for-url image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" volumeMounts: - mountPath: "/tmp" name: workdir env: - name: multipleUrls valueFrom: configMapKeyRef: name: some-config key: multipleUrls command: - '/bin/bash' - '-c' - | echo "Actual Data: $multipleUrls" : : Logic to find the valid url and store it in the /tmp/ip.env file : : echo "Accessible Url is:$hosturl" echo $hosturl> /tmp/ip.env if [ -s /tmp/ip.env ]; then echo "HOSTURL is: " `cat /tmp/ip.env` exit 0; else sleep 10 exit 1; fi volumes: - name: workdir emptyDir: {}
kubectl describe pod 输出
Name: some-name-nckfs Namespace: default Priority: 0 Node: 10.22.44.169/10.22.44.169 Start Time: Sat, 29 Oct 2022 15:11:11 +0000 Labels: controller-revision-hash=55ff545b89 pod-template-generation=32 Annotations: <none> Status: Running IP: 10.22.44.169 IPs: IP: 10.22.44.169 Controlled By: DaemonSet/some-something Init Containers: wait-for-hosturl: Container ID: docker://62e3b20b6eeeca4f37e78c02fa8d68b482bde1644fc30a0ed94f788c35d3cd86 Image ID: docker://sha256:805f01431e534713510826a63dab75873d45aa96c6a9614f2c9bfbeab800a774 Port: <none> Host Port: <none> Command: /bin/bash -c :: :: State: Terminated Reason: Completed Exit Code: 0 Started: Sat, 29 Oct 2022 15:11:12 +0000 Finished: Sat, 29 Oct 2022 15:11:12 +0000 Ready: True Restart Count: 0 Environment: multipleUrls: <set to the key 'multipleUrls' of config map 'some-config'> Optional: false Mounts: /tmp from workdir (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-vks68 (ro) Containers: some-something: Container ID: docker://1fd6d60a70a6e2c10339809e4bb3084738aee7500dc0087973801c1c3a5a497a Image ID: docker://sha256:805f01431e534713510826a63dab75873d45aa96c6a9614f2c9bfbeab800a774 Port: 9091/TCP Host Port: 9091/TCP Command: sh -c env HOST_URL=$(cat /tmp/ip.env) State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Completed Exit Code: 0 Started: Sat, 29 Oct 2022 15:57:42 +0000 Finished: Sat, 29 Oct 2022 15:57:42 +0000 Ready: False Restart Count: 14 Environment: Mounts: /tmp from workdir (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-vks68 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: workdir: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> default-token-vks68: Type: Secret (a volume populated by a Secret) SecretName: default-token-vks68 Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/disk-pressure:NoSchedule op=Exists node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/network-unavailable:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists node.kubernetes.io/pid-pressure:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists node.kubernetes.io/unschedulable:NoSchedule op=Exists Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Created 49m kubelet Created container wait-for-hosturl Normal Started 49m kubelet Started container wait-for-hosturl Normal Created 48m (x5 over 49m) kubelet Created container some-something Normal Started 48m (x5 over 49m) kubelet Started container some-something Warning BackOff 4m57s (x212 over 49m) kubelet Back-off restarting failed container
kubectl logs 输出
KUBERNETES_SERVICE_PORT=443 KUBERNETES_PORT=tcp://10.43.0.1:443 HOSTNAME=something HOME=/root KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin KUBERNETES_PORT_443_TCP_PORT=443 KUBERNETES_PORT_443_TCP_PROTO=tcp KUBERNETES_PORT_443_TCP=tcp://10.43.0.1:443 KUBERNETES_SERVICE_PORT_HTTPS=443 KUBERNETES_SERVICE_HOST=10.43.0.1 JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 PWD=/ HOST_URL=10.22.44.168
原因分析
主容器的command仅执行了env HOST_URL=$(cat /tmp/ip.env),该命令执行完成后立即退出(返回码0),而Kubernetes容器需要长期运行的进程才能保持存活状态。原配置未指定command,会使用镜像默认的启动命令维持容器运行;修改后覆盖了默认命令,导致容器执行完env命令就终止,kubelet不断重启容器,最终出现CrashLoopBackOff。
解决方案
需要在设置环境变量后,执行镜像原本的启动命令,确保容器长期运行。有两种常见方式:
方式1:直接拼接命令
假设镜像默认启动命令为java -jar app.jar(需根据实际镜像调整),修改主容器的command:
command: - sh - -c - "export HOST_URL=$(cat /tmp/ip.env) && exec java -jar app.jar"
使用exec让原启动进程成为容器的PID 1,确保信号能正确传递。
方式2:通过环境变量文件加载
先修改initContainer的写入逻辑,将环境变量以KEY=VALUE格式写入文件:
echo "HOST_URL=$hosturl" > /tmp/ip.env
然后主容器通过source加载环境变量并执行原命令:
command: - sh - -c - "source /tmp/ip.env && exec <原镜像启动命令>"
内容的提问来源于stack exchange,提问作者Nagaraju Chitimilla
相关产品推荐
相关产品推荐

