创建StatefulSet报Pod未绑定即时PersistentVolumeClaims排查
StatefulSet Pod 报未绑定PVC故障排查
pod has unbound immediate PersistentVolumeClaims报错的直接原因是Pod声明挂载的PVC找不到匹配的PV完成绑定,结合你贴的配置,按以下顺序排查即可解决:
- 校验引用的StorageClass是否存在
你的StatefulSet在volumeClaimTemplates里写死了storageClassName: slow,先执行命令确认这个存储类在集群里真实存在:
如果返回NotFound,说明根本没有这个名为slow的存储类,PVC没法自动创建PV,自然绑定失败。两种修复方式选一个就行:kubectl get sc slow- 集群中创建名为slow的StorageClass,对接可用的存储后端(NFS、本地盘、云盘都可以),配置对应的provisioner实现PV自动动态创建
- 修改StatefulSet配置,把
storageClassName换成集群里已经存在的存储类名,直接删掉这个字段就会使用集群默认的StorageClass
- 检查StorageClass的卷绑定模式
如果slow存储类确实存在,执行kubectl describe sc slow查看VolumeBindingMode配置项:- 如果值是
Immediate:PVC一创建就会立刻找PV绑定,如果你用的是节点本地存储,这时候Pod还没调度到具体节点,找不到对应节点上的本地PV就会绑定失败,把绑定模式改成WaitForFirstConsumer即可,等Pod调度到对应节点之后再绑定PV,刚好适配本地存储的节点拓扑限制 - 如果值已经是
WaitForFirstConsumer,继续往下排查
- 如果值是
- 静态供给场景手动创建匹配PV
如果你没配置存储动态供给插件,需要手动创建PV匹配StatefulSet自动生成的PVC。StatefulSet会给每个副本生成独立PVC,命名规则是<卷声明模板名>-<StatefulSet名>-<副本序号>,你这套3副本的配置会生成slow-phponcio-0、slow-phponcio-1、slow-phponcio-2三个PVC,手动创建的PV必须同时满足以下条件才能和PVC绑定:- PV的
storageClassName必须是slow - PV容量不能小于PVC申请的1Gi
- PV的访问模式必须包含
ReadWriteOnce - PV没有被其他PVC提前绑定占用
- PV的
- 清理干扰资源
你贴的第一个Pod是之前手动创建的独立Pod,标签是run=phponcio,既没有StatefulSet管理的app=phponcio标签,也没挂载slow卷,和这次故障完全没关系,直接执行kubectl delete pod phponcio删掉即可,避免排查时混淆资源。
附相关资源信息
异常StatefulSet详情
Name: phponcio Namespace: default CreationTimestamp: Thu, 02 Jun 2022 10:35:08 +0200 Selector: app=phponcio Labels: <none> Annotations: <none> Replicas: 3 desired | 1 total Update Strategy: RollingUpdate Partition: 0 Pods Status: 0 Running / 1 Waiting / 0 Succeeded / 0 Failed Pod Template: Labels: app=phponcio Containers: phponcio: Image: 84d3f7ba5876/php-apache-mysql Port: 80/TCP Host Port: 0/TCP Environment: <none> Mounts: /usr/share from slow (rw) Volumes: <none> Volume Claims: Name: slow StorageClass: slow Labels: <none> Annotations: <none> Capacity: 1Gi Access Modes: [ReadWriteOnce] Events: <none>
提交的资源YAML
apiVersion: v1 kind: Service metadata: name: phponcio labels: app: phponcio spec: ports: - port: 80 name: web clusterIP: None selector: app: phponcio --- apiVersion: apps/v1 kind: StatefulSet metadata: name: phponcio spec: selector: matchLabels: app: phponcio serviceName: phponcio replicas: 3 template: metadata: labels: app: phponcio spec: terminationGracePeriodSeconds: 10 containers: - name: phponcio image: 84d3f7ba5876/php-apache-mysql ports: - containerPort: 80 name: phponcio volumeMounts: - name: slow mountPath: /usr/share volumeClaimTemplates: - metadata: name: slow spec: accessModes: [ "ReadWriteOnce" ] storageClassName: slow resources: requests: storage: 1Gi
集群中存在的无关旧Pod详情
Name: phponcio Namespace: default Priority: 0 Node: juanaraque-worknode/10.0.0.2 Start Time: Wed, 01 Jun 2022 13:49:08 +0200 Labels: run=phponcio Annotations: cni.projectcalico.org/containerID: 768caa8f4e9b0683033e279aa259d6673d81fe38b62a2ea06c7d256805d522f0 cni.projectcalico.org/podIP: 192.168.240.69/32 cni.projectcalico.org/podIPs: 192.168.240.69/32 Status: Running IP: 192.168.240.69 IPs: IP: 192.168.240.69 Containers: phponcio: Container ID: docker://75f90344ada396be965f0ae5c33bb7ea8c110d9ea1e014b75f36263b9cd72504 Image: 84d3f7ba5876/php-apache-mysql Image ID: docker-pullable://84d3f7ba5876/php-apache-mysql@sha256:bf0cd01ae4b77cca146dcd54d8a447ba8b6c7f5a9e11d6dab6d19f429fb111d1 Port: <none> Host Port: <none> State: Running Started: Wed, 01 Jun 2022 14:31:53 +0200 Last State: Terminated Reason: Error Exit Code: 255 Started: Wed, 01 Jun 2022 13:54:26 +0200 Finished: Wed, 01 Jun 2022 14:30:44 +0200 Ready: True Restart Count: 2 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bld5b (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: kube-api-access-bld5b: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: <none>
内容的提问来源于stack exchange,提问作者Jose Matinez
相关产品推荐
相关产品推荐

