GKE上的Elasticsearch：无法使用GCS插件配置快照与恢复

Got it, let's walk through this step by step—since you're running ES on GKE with a mix of Deployments and StatefulSets, we need to handle the keystore setup securely and in a way that works across all your node types.

1. First, Store Your GCP Service Account Key as a Kubernetes Secret #

Instead of hardcoding or copying the key directly into pods, we'll use a Kubernetes Secret to safely pass the key to your ES nodes:

kubectl create secret generic gcp-es-snapshot-key \
  --from-file=service-account.json=/path/to/your/local/gcp-key.json

This creates a secret named gcp-es-snapshot-key that holds your JSON key file with restricted permissions.

2. Update Your ES Deployments/StatefulSets to Mount the Secret #

You need to modify the YAML for your Master Deployment, Client Deployment, and Data StatefulSet to mount this secret into the ES pods. Here's what to add:

Add Volumes Section #

Under spec.template.spec.volumes, add:

- name: gcp-snapshot-key
  secret:
    secretName: gcp-es-snapshot-key
    defaultMode: 0400  # Restrict permissions to read-only for the owner

Add VolumeMounts Section #

Under spec.template.spec.containers[name: elasticsearch].volumeMounts, add:

- name: gcp-snapshot-key
  mountPath: /usr/share/elasticsearch/config/gcp
  readOnly: true

This mounts the secret's contents into /usr/share/elasticsearch/config/gcp inside the pod—safe and read-only.

For your Data StatefulSet, make sure you're also persisting the ES config directory (where the keystore lives) to your PV. If you haven't already, add a volume mount for /usr/share/elasticsearch/config pointing to your existing PV or a dedicated subpath.

3. Populate the Elasticsearch Keystore (Automated with Init Containers) #

Instead of manually exec-ing into pods, use an init container to set up the keystore before the ES container starts. This ensures consistency across all nodes.

Add this init container to your Deployment/StatefulSet YAML under spec.template.spec.initContainers:

- name: setup-keystore
  image: docker.elastic.co/elasticsearch/elasticsearch:<your-es-version>
  command:
    - bash
    - -c
    - |
      # Create keystore if it doesn't exist
      if [ ! -f /usr/share/elasticsearch/config/elasticsearch.keystore ]; then
        bin/elasticsearch-keystore create
      fi
      # Add GCP credentials to keystore
      bin/elasticsearch-keystore add-file gcs.client.default.credentials_file /usr/share/elasticsearch/config/gcp/service-account.json
      # Ensure permissions are correct for ES user (uid 1000)
      chown -R 1000:0 /usr/share/elasticsearch/config
  volumeMounts:
    - name: gcp-snapshot-key
      mountPath: /usr/share/elasticsearch/config/gcp
      readOnly: true
    # Mount the ES config directory (use your existing PV mount here for StatefulSets)
    - name: es-config
      mountPath: /usr/share/elasticsearch/config

Replace <your-es-version> with your actual Elasticsearch version (e.g., 8.10.2), and es-config with the name of your volume that holds the ES config (for StatefulSets, this should be your PV volume).

4. Roll Out the Updates #

Apply your modified YAML files and restart the pods to pick up the new config:

# For Master/Client Deployments
kubectl apply -f es-master-deployment.yaml
kubectl rollout restart deployment/es-master

# For Data StatefulSet
kubectl apply -f es-data-statefulset.yaml
kubectl rollout restart statefulset/es-data

5. Configure the GCS Snapshot Repository #

Once all pods are running, exec into any ES pod (Master or Client works) to create the snapshot repository:

kubectl exec -it <es-pod-name> -- curl -XPUT "http://localhost:9200/_snapshot/gcs_backup" \
  -H 'Content-Type: application/json' \
  -d'{
    "type": "gcs",
    "settings": {
      "bucket": "your-gcs-bucket-name",  # Replace with your bucket name
      "client": "default"
    }
  }'

Verify the Repository #

Check that the repository is working:

kubectl exec -it <es-pod-name> -- curl -XGET "http://localhost:9200/_snapshot/gcs_backup/_verify"

You should get a response indicating the repository is valid.

Key Notes to Remember #

• GCP Permissions: Ensure your service account has the necessary GCS permissions (storage.objects.create, storage.objects.list, storage.objects.get, storage.objects.delete) on your backup bucket.
• X-Pack Security: If you're using ES security, make sure your admin user has the manage_snapshots cluster privilege.
• Keystore Persistence: For StatefulSets, since we're mounting the config directory to a PV, the keystore will persist across pod restarts. For Deployments, if pods are recreated, the init container will re-populate the keystore automatically.

内容的提问来源于stack exchange，提问作者spider