K8s Pod无全权限访问NFS动态PV目录致API超时问题

已通过StorageClass动态供应Pod，配置了Pod、PVC及nfs-client-provisioner，但Pod运行时API访问/修改PV文件超时。PV目录已设全权限，但生成的文件权限为-rw-r--r--，需改为-rwxrwxrwx才能满足API调用需求。

当前文件权限：

-rw-r--r--. 1 root root    178688 Jul 16 19:29 data00104a.tar
-rw-r--r--. 1 root root    160768 Jul 16 19:45 data00105a.tar

目标权限：

-rwxrwxrwx. 1 root root    178688 Jul 16 19:29 data00104a.tar
-rwxrwxrwx. 1 root root    160768 Jul 16 19:45 data00105a.tar

相关配置文件 #

pod.yaml #

kind: Deployment
metadata:
  name: dms
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dms
  template:
    metadata:
      labels:
        app: dms
    spec:
      volumes:
        - name: dms-repo-storage
          persistentVolumeClaim:
            claimName: dms-repo
        - name: dms-log-storage
          persistentVolumeClaim:
            claimName: dms-log
      containers:
      - name: dms
        image: xxxx/xxx/dms:3
        env:
        - name: PORT
          value: "61024"
        - name: QUARKUS_LOG_FILE_PATH
          value: "/log/applicationPassive.log"
        - name: REPO_HOME_DIR
          value: "/document/repo"
        - name: REPO_TYPE
          value: "SEGMENT"
        ports:
        - containerPort: 61024
        volumeMounts:
        - mountPath: "/document/repo"
          name: dms-repo-storage
        - mountPath: "/log"
          name: dms-log-storage
---
apiVersion: v1
kind: Service
metadata:
  name: dms
  labels:
    app: dms
spec:
  ports:
  - port: 61024
    name: http
    targetPort: 61024
  selector:
    app: dms
  type: ClusterIP

PVC.yaml #

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dms-repo
spec:
  storageClassName: nfs-storage-app
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20Gi

nfs-client-deployment.yaml #

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner-app
  namespace: msdev1
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner-app
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner-app
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner-app
    namespace: msdev1
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner-app
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner-app
  namespace: msdev1
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner-app
  namespace: msdev1
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner-app
    namespace: msdev1
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner-app
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage-app
provisioner: nfs-provisioner-app
parameters:
  archiveOnDelete: "true"
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-client-provisioner-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner-app
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner-app
    spec:
      serviceAccountName: nfs-client-provisioner-app
      containers:
        - name: nfs-client-provisioner-app
          image: xxx/xxxx/nfs-client-provisioner:v3.1.0-k8s1.11
          volumeMounts:
            - name: nfs-client-root-app
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: nfs-provisioner-app
            - name: NFS_SERVER
              value: "serveripaddress"
            - name: NFS_PATH
              value: /nfs_srims
      volumes:
        - name: nfs-client-root-app
          nfs:
            server: "serveripaddress"
            path: /nfs_srims

解决方案 #

1. 修改StorageClass配置，添加权限参数 #

在nfs-client-deployment.yaml的StorageClass部分，添加挂载选项和默认权限配置：

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage-app
provisioner: nfs-provisioner-app
parameters:
  archiveOnDelete: "true"
  defaultMode: "0777" # 动态供应PV的默认文件权限
mountOptions:
  - umask=000 # 挂载时设置umask，新文件默认权限为777

2. 配置Pod的SecurityContext，确保容器权限 #

修改pod.yaml的Deployment模板，添加SecurityContext设置，让容器以root用户运行并设置文件组：

kind: Deployment
metadata:
  name: dms
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dms
  template:
    metadata:
      labels:
        app: dms
    spec:
      securityContext:
        fsGroup: 0 # 挂载卷的文件组设为root
        runAsUser: 0 # Pod以root用户运行
      volumes:
        - name: dms-repo-storage
          persistentVolumeClaim:
            claimName: dms-repo
        - name: dms-log-storage
          persistentVolumeClaim:
            claimName: dms-log
      containers:
      - name: dms
        image: xxxx/xxx/dms:3
        securityContext:
          runAsUser: 0 # 容器以root用户运行
          runAsGroup: 0
        env:
        - name: PORT
          value: "61024"
        - name: QUARKUS_LOG_FILE_PATH
          value: "/log/applicationPassive.log"
        - name: REPO_HOME_DIR
          value: "/document/repo"
        - name: REPO_TYPE
          value: "SEGMENT"
        ports:
        - containerPort: 61024
        volumeMounts:
        - mountPath: "/document/repo"
          name: dms-repo-storage
        - mountPath: "/log"
          name: dms-log-storage

3. 生效步骤 #

• 删除现有PVC和PV：

  kubectl delete pvc dms-repo dms-log
  kubectl delete pv <对应PV名称>
  

• 重新部署组件：

  kubectl apply -f nfs-client-deployment.yaml
  kubectl apply -f pod.yaml
  

• 验证权限：

  kubectl exec -it <dms-pod名称> -- bash
  touch /document/repo/test-file
  ls -l /document/repo/test-file
  

  输出应为-rwxrwxrwx. 1 root root ... test-file

内容的提问来源于stack exchange，提问作者Prabath