AWS Secrets Manager与EKS集成挂载失败问题求助

问题场景 #

已完成AWS Secrets Manager CSI驱动和AWS Provider的安装，部署应用时出现挂载失败，相关配置及错误日志如下：

部署配置YAML #

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: profile-deployment
  namespace: tsc-sandbox-deployments
  labels:
    app: smoothie
    svc: profile
spec:
  replicas: 2
  selector:
    matchLabels:
      app: smoothie
      svc: profile
  template:
    metadata:
      labels:
        app: smoothie
        svc: profile
    spec:
      serviceAccountName: profile-sa
      volumes:
      - name: secrets-store-inline
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: profile-secrets-sandbox
      containers:
      - name: profile
        image: ${ECRREGISTRY}/profile:${BITBUCKET_BUILD_NUMBER}
        imagePullPolicy: Always
        env:
        - name: spring.profiles.active
          value: ${ENVPROFILE}
        - name: app.olo.url
          valueFrom:
            secretKeyRef:
              key: STAGE_TSC_APP_OLO_URL
              name: profile-sandbox-secrets
        
        ports:
        - containerPort: 8097
        volumeMounts:
        - name: secrets-store-inline
          mountPath: "/mnt/secrets-store"
          readOnly: true
        resources:
          limits:
            cpu: 500m
          requests:
            cpu: 500m
        readinessProbe:
          httpGet:
            path: /profile/actuator/health/readiness
            port: 8097
          initialDelaySeconds: 100
          periodSeconds: 5
          successThreshold: 1
          failureThreshold: 3
        lifecycle:
          preStop:
            exec:
              command: ["/bin/bash", "-c", "sleep 90"]
      terminationGracePeriodSeconds: 30
  strategy:
      type: RollingUpdate
      rollingUpdate:
        maxSurge: 1
        maxUnavailable: 1
---
apiVersion: v1
kind: Service
metadata:
  name: profile-svc
  namespace: tsc-sandbox-deployments
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /profile/actuator/health/readiness
spec:
  selector:
    app: smoothie
    svc: profile
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8097

---
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: profile-secrets-sandbox
  namespace: tsc-sandbox-deployments
spec:
  provider: aws
  parameters:    
    objects: |
      - objectName: "arn:aws:secretsmanager:us-east-2:xxxxxxxxxxxxxxxxxxx"
        objectAlias: "profileSandbox"
        objectType: "secretsmanager"
        jmesPath:
          - path: APP_OLO_URL
            objectAlias: TSC_APP_OLO_URL
  secretObjects:
    - secretName: profile-sandbox-secrets
      type: Opaque
      type: Opaque
      data:
        - key: STAGE_TSC_APP_OLO_URL
          objectName: TSC_APP_OLO_URL

错误事件日志 #

Events:
Type     Reason       Age                From               Message
---
Normal   Scheduled    43s                default-scheduler  Successfully assigned tsc-sandbox-deployments/profile-deployment-6d7c7c75f-dvxdf to ip-xxxxxxx.us-east-2.compute.internal
Warning  FailedMount  10s (x7 over 43s)  kubelet            MountVolume.SetUp failed for volume "secrets-store-inline" : rpc error: code = Unknown desc = failed to mount secrets store objects for pod tsc-sandbox-deployments/profile-deployment-6d7c7c75f-dvxdf, err: rpc error: code = Unknown desc = Failed to fetch secret from all regions: profileSandbox

解决步骤 #

• 补全Secrets Manager ARN
  当前配置中的objectName ARN不完整，正确格式应为arn:aws:secretsmanager:us-east-2:ACCOUNT_ID:secret:SECRET_NAME-XXXX（末尾随机后缀由AWS自动生成）。替换为目标Secret的完整ARN，确保能精准定位到对应Secret。

• 验证ServiceAccount权限配置

  1. 确认profile-sa对应的IAM角色拥有secretsmanager:GetSecretValue权限，策略需包含目标Secret的完整ARN，示例策略：

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": "secretsmanager:GetSecretValue",
                 "Resource": "完整的Secret ARN"
             }
         ]
     }
     

  2. 检查ServiceAccount是否通过IRSA绑定了正确的IAM角色，确保ServiceAccount资源中存在注解eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/IAM_ROLE_NAME。

• 修复SecretProviderClass的配置错误
  删除secretObjects部分重复的type: Opaque行，避免无效配置干扰解析。修改后的secretObjects段如下：

  secretObjects:
    - secretName: profile-sandbox-secrets
      type: Opaque
      data:
        - key: STAGE_TSC_APP_OLO_URL
          objectName: TSC_APP_OLO_URL
  

• 确认区域一致性
  确保EKS集群所在区域与Secrets Manager的区域（us-east-2）一致，避免跨区域访问引发的连接或权限问题。

• 查看CSI驱动日志定位细节
  运行以下命令获取CSI驱动及AWS Provider的日志，排查具体错误原因：

  kubectl logs -n kube-system -l app=secrets-store-csi-driver
  kubectl logs -n kube-system -l app=csi-secrets-store-provider-aws
  

  日志中会包含更具体的错误信息，比如权限不足、Secret不存在等，帮助进一步定位问题。

内容的提问来源于stack exchange，提问作者Deepali Verma