AWS EKS部署StatefulSet遇PVC绑定失败问题求助

问题描述 #

在AWS EKS部署3副本StatefulSet，要求每个副本动态创建独立EFS存储，EFS地址存于AWS Secrets Manager，部署后出现调度错误：

Warning FailedScheduling 4m58s default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.

提供的配置文件 #

secrets-provider-class.yaml #

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-secrets-provider
spec:
  provider: aws
  parameters:
    objects:  |
      - objectName: "efs-server-address"
        objectType: secretsmanager
        objectVersion: ""

efs-storageclass.yaml #

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-storageclass
provisioner: secrets-store.csi.k8s.io
parameters:
  secretProviderClass: "aws-secrets-provider"

eks-secrets-manager-efs-sa.yaml #

apiVersion: v1
kind: ServiceAccount
metadata:
  name: eks-secrets-manager-efs-sa
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::1234567890:policy/eks-secrets-manager-efs-policy

efs-pvc.yaml #

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-pvc
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-storageclass
  resources:
    requests:
      storage: 1Gi
  selector:
      matchLabels:
        fileSystemId: fs-0f012ab34567ab89c

statefulset-01.yaml #

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-01
  labels:
    app: app-sample-01
spec:
  replicas: 3
  selector:
    matchLabels:
      app: app-sample-01
  template:
    metadata:
      labels:
        app: app-sample-01
    spec:
      serviceAccountName: eks-secrets-manager-efs-sa
      containers:
        - name: container-sample-01
          image: '123456789000.dkr.ecr.us-east-1.amazonaws.com/sample-repo:latest'
          ports:
            - containerPort: 8080
          volumeMounts:
            - name: efs-volume
              mountPath: /data
  volumeClaimTemplates:
      - metadata:
          name: efs-volume
        spec:
          accessModes:
            - ReadWriteMany
          storageClassName: efs-storageclass
          resources:
            requests:
              storage: 1Gi

错误根源与修复步骤 #

1. StorageClass Provisioner 配置错误 #

secrets-store.csi.k8s.io是用于挂载密钥的CSI驱动，不能作为EFS存储的Provisioner，必须使用EFS专用的efs.csi.aws.com。

修复后的efs-storageclass.yaml：

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-storageclass
provisioner: efs.csi.aws.com
parameters:
  provisioningMode: efs-ap  # 动态创建EFS访问点模式
  fileSystemId: fs-0f012ab34567ab89c  # 替换为你的EFS文件系统ID
  directoryPerms: "700"
reclaimPolicy: Retain

2. ServiceAccount 注解错误 #

eks.amazonaws.com/role-arn必须指向IAM角色ARN，而非策略ARN。需要先创建一个IAM角色，关联eks-secrets-manager-efs-policy策略，再将角色ARN填入注解。

修复后的eks-secrets-manager-efs-sa.yaml：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: eks-secrets-manager-efs-sa
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::1234567890:role/eks-secrets-manager-efs-role  # 替换为实际IAM角色ARN

3. 缺少EFS CSI驱动部署 #

确保EFS CSI驱动已安装到集群：

kubectl apply -k "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=release-1.7"

4. Secrets Manager 集成逻辑错误 #

Secrets Store CSI Driver无法直接为StorageClass提供EFS地址参数。若需从Secrets Manager读取EFS地址，可通过以下方式：

• 将Secrets Manager中的EFS地址同步为K8s Secret（使用External Secrets Operator）
• 在Pod中挂载Secrets Store CSI卷，让应用直接读取EFS地址（但此方式不适用于StorageClass参数）

简化方案：直接在StorageClass中指定EFS文件系统ID（如修复步骤1所示），无需通过Secrets Manager传递，更符合EFS CSI驱动的使用逻辑。

5. 冗余PVC与错误选择器 #

• efs-pvc.yaml为冗余配置，StatefulSet的volumeClaimTemplates会自动为每个副本生成独立PVC，无需手动创建。
• PVC中的selector标签匹配逻辑无效，动态Provision的PV不会带有fileSystemId标签，需移除该配置。

6. 验证部署顺序 #

按以下顺序重新部署资源：

# 部署ServiceAccount
kubectl apply -f eks-secrets-manager-efs-sa.yaml
# 部署StorageClass
kubectl apply -f efs-storageclass.yaml
# 部署StatefulSet
kubectl apply -f statefulset-01.yaml

验证状态：

# 查看PVC绑定状态
kubectl get pvc
# 查看Pod调度状态
kubectl get pods

内容的提问来源于stack exchange，提问作者MChiru