Hey there! Let's walk through configuring storage volumes and ensuring high availability for your 4-node Minio cluster deployed via Helm on Kubernetes.

1. Storage Volume Configuration for Distributed Minio #

Distributed Minio relies on each pod having its own persistent storage volume (since erasure coding splits data across multiple drives). Here's how to manage this effectively:

Check Current PVCs #

First, verify the existing persistent volume claims (PVCs) tied to your Minio pods:

kc get pvc -n digimops

You should see 4 PVCs (one per pod) with Bound status if everything's working correctly.

Customize Storage via Helm Values #

To adjust storage settings (like size, storage class, or access mode), create a custom values.yaml file with the persistence section:

persistence:
  enabled: true
  size: 100Gi  # Adjust based on your storage needs
  storageClass: "your-storage-class"  # Replace with your cluster's storage class (e.g., gp3 for EBS)
  accessMode: ReadWriteOnce  # Standard for single-node access (each pod uses its own PVC)
  volumeName: ""  # Leave empty to let Kubernetes auto-provision

For distributed mode, the Minio Helm chart automatically creates one PVC per pod, so you don't need to define individual volumes here.

Apply Changes with Helm Upgrade #

Update your deployment to apply the new storage configuration:

helm upgrade minio-cluster minio/minio -n digimops -f custom-values.yaml

Wait for the pods to restart and confirm PVCs are re-bound correctly.

Alternative: Local/HostPath Storage (Testing Only) #

If you need to use local node storage (not recommended for production), add this to your values.yaml:

persistence:
  enabled: true
  type: hostPath
  hostPath: "/mnt/minio-storage"  # Ensure this directory exists on all nodes with proper permissions

2. High Availability (HA) Setup for Distributed Minio #

Your 4-node cluster is already a foundation for HA (Minio uses erasure coding with 2 data and 2 parity shards for 4 drives), but we can harden it further:

Ensure Pods Spread Across Nodes #

Use pod anti-affinity to prevent multiple Minio pods from running on the same node (critical for fault tolerance if a node fails):
Add this to your values.yaml:

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
        - key: app.kubernetes.io/name
          operator: In
          values:
          - minio
      topologyKey: kubernetes.io/hostname

This forces Kubernetes to schedule each Minio pod on a unique node.

Configure a Load Balanced Service #

Make sure your Minio service is set to LoadBalancer (or use an Ingress for HTTP/HTTPS access) so clients can connect to any pod seamlessly:

service:
  type: LoadBalancer
  port: 9000
  targetPort: 9000

After upgrading, get the external IP with:

kc get svc minio-cluster -n digimops

Use this IP to access Minio; the service will distribute traffic across all pods.

Verify HA Functionality #

Test fault tolerance by deleting one pod and confirming data remains accessible:

kc delete pod minio-cluster-0 -n digimops

Wait for the pod to restart, then use the Minio CLI (mc) or console to check if your buckets/data are still available. Minio will automatically recover using erasure coding once the pod is back online.

Enable Minio Console for Monitoring #

Add this to values.yaml to enable the web console (great for checking cluster status):

console:
  enabled: true
  service:
    type: LoadBalancer
    port: 9090

Access the console via its external IP to monitor pod health, storage usage, and HA status.

Key Notes #

• Erasure Coding: For 4 nodes, Minio uses 2 data and 2 parity shards—you can lose up to 2 pods without losing data.
• Storage Class: Use a high-performance, durable storage class (like AWS EBS gp3, GCP Persistent Disk, or local SSDs) for production.
• Backups: Even with HA, regularly back up critical data using Minio's mc mirror command or a third-party tool.

内容的提问来源于stack exchange，提问作者Babu Natesan