Kubernetes v1.17.2多租户环境动态PV持久化最佳实践咨询

Hey Shane, great question—this is a super common pattern for multi-tenant Kubernetes setups where you need per-tenant data persistence through Pod restarts or updates. Let’s walk through the best practices and concrete implementation steps tailored to your Kubernetes v1.17.2 environment.

The key here is to bind each tenant’s persistent storage to a unique tenant identifier (like user-1, user-2). By linking your Deployment, PersistentVolumeClaim (PVC), and Pod environment variables to this identifier, you ensure that when a Pod is deleted and recreated, it will reattach to the exact same dynamically provisioned PersistentVolume (PV) and retain the tenant’s data.

1. Configure a Dynamic Provisioning StorageClass #

First, you need a StorageClass to enable dynamic PV creation. This tells Kubernetes how to provision storage (e.g., AWS EBS, GCP Persistent Disk, or local storage) when a PVC is created. For v1.17.2, use the storage.k8s.io/v1 API version (avoid deprecated v1beta1):

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: tenant-storage
provisioner: kubernetes.io/aws-ebs # Replace with your cloud provider's provisioner or local-path-provisioner
parameters:
  type: gp2 # Adjust storage type based on your needs
reclaimPolicy: Retain # Critical: Prevents PV deletion if the PVC is accidentally removed
volumeBindingMode: Immediate

2. Create Tenant-Specific PVCs #

For each tenant, create a dedicated PVC that includes their identifier in the name and labels. This ensures the PVC is tied exclusively to that tenant:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-db-user-1 # Include tenant ID in the name
  labels:
    tenant: user-1 # Add a tenant label for easy filtering/management
spec:
  storageClassName: tenant-storage
  accessModes:
    - ReadWriteOnce # Suitable for single-database Pods
  resources:
    requests:
      storage: 10Gi # Adjust storage size per tenant

Repeat this for every tenant (e.g., pvc-db-user-2, pvc-db-user-3, etc.). For large numbers of tenants, use tools like Helm or Kustomize to automate this instead of manual creation.

3. Link PVCs to Tenant Deployments #

Update your webserver and database Deployment manifests to:

• Mount the tenant’s specific PVC
• Pass the tenant ID as an environment variable (for application logging, metrics, or logic)

Here’s an example for a tenant user-1’s database Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: db-user-1
  labels:
    tenant: user-1
    app: db
spec:
  replicas: 1
  selector:
    matchLabels:
      tenant: user-1
      app: db
  template:
    metadata:
      labels:
        tenant: user-1
        app: db
    spec:
      containers:
      - name: db
        image: mysql:5.7
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: your-secure-password
        - name: TENANT_ID # Pass tenant ID to the application
          value: user-1
        volumeMounts:
        - name: db-storage
          mountPath: /var/lib/mysql # Path where the database stores data
      volumes:
      - name: db-storage
        persistentVolumeClaim:
          claimName: pvc-db-user-1 # Bind to the tenant's PVC

The webserver Deployment follows the same pattern—if it needs persistent storage (e.g., user uploads), create a corresponding pvc-web-user-1 and mount it similarly.

• Use Labels for Tenant Resource Management: Add a tenant: user-x label to all resources (Deployments, Services, PVCs, Pods) belonging to a tenant. This makes it easy to query, update, or delete all resources for a tenant with commands like kubectl get pods -l tenant=user-1.
• Isolate Tenants with Namespaces (Optional but Recommended): For stronger isolation, create a dedicated Namespace for each tenant (e.g., tenant-user-1). This prevents resource name collisions and lets you apply tenant-specific RBAC policies.
• Automate Resource Creation with Helm/Kustomize: Manually creating resources for every tenant is error-prone. Use a Helm chart with the tenant ID as a parameter, or Kustomize overlays, to generate all tenant resources in one go.
• Set PVC Reclaim Policy to Retain: As shown in the StorageClass example, this ensures that if a PVC is deleted, the underlying PV (and its data) is preserved. You can manually reclaim the PV later if needed.
• Consider StatefulSets for Databases: While Deployments work for single-instance databases, StatefulSets are designed for stateful workloads. They provide stable network identities and guaranteed PVC binding, which is useful if you ever need to scale the database (though your current setup uses 1:1 tenant-db Pods, this is a future-proofing step).

• Avoid Deprecated APIs: Stick to apps/v1 for Deployments and storage.k8s.io/v1 for StorageClasses—v1beta1 versions were deprecated in v1.16 and removed in later versions.
• Verify Dynamic Provisioning Support: Ensure your cluster has the correct provisioner installed (e.g., cloud provider’s provisioner or local-path-provisioner for on-prem clusters) to support dynamic PV creation.
• Check Volume Binding Mode: Immediate binding mode works for most cases, but if you need to delay binding until a Pod is scheduled (e.g., for local storage), use WaitForFirstConsumer (supported in v1.17+).

内容的提问来源于stack exchange，提问作者Shane