Kubernetes部署etcd备份Job时YAML数据校验报错排查

问题场景 #

在进行Kubernetes实验练习时，执行创建etcd备份Job的步骤，需要通过Job资源启动Pod，运行etcdctl snapshot save命令完成集群etcd数据备份。编写snapshot.yaml后怀疑存在缩进格式错误，查阅相关文档后仍未定位问题点。

原始配置与报错信息 #

原始snapshot.yaml内容 #

apiVersion: batch/v1
kind: Job
metadata:
  name: backup
  namespace: management
spec:
  template:
    spec:
      containers:
      # Use etcdctl snapshot save to create a snapshot in the /snapshot directory
      - command:
        - /bin/sh
        args:
        - -ec
        - etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key snapshot save /snapshots/backup.db
        # The same image used by the etcd pod
        image: k8s.gcr.io/etcd-amd64:3.1.12
        name: etcdctl
        env:
        # Set the etcdctl API version to 3 (to match the version of etcd installed by kubeadm)
        - name: ETCDCTL_API
          value: '3'
        volumeMounts:
        - mountPath: /etc/kubernetes/pki/etcd
          name: etcd-certs
          readOnly: true
        - mountPath: /snapshots
          name: snapshots
      # Use the host network where the etcd port is accessible (etcd pod uses hostnetwork)
      # This allows the etcdctl to connect to etcd that is listening on the host network
      hostNetwork: true
      affinity:
        # Use node affinity to schedule the pod on the master (where the etcd pod is)
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node-role.kubernetes.io/master
                operator: Exists
      restartPolicy: OnFailure
      tolerations:
      # tolerate the master's NoSchedule taint to allow scheduling on the master
      - effect: NoSchedule
        operator: Exists
      volumes:
      # Volume storing the etcd PKI keys and certificates
      - hostPath:
        path: /etc/kubernetes/pki/etcd
        type: DirectoryOrCreate
        name: etcd-certs
      # A volume to store the backup snapshot
      - hostPath:
        path: /snapshots
        type: DirectoryOrCreate
        name: snapshots

执行创建命令返回报错 #

运行命令：kubectl create -f snapshot.yaml
返回错误信息：

johsttin@umasternode:~$ kubectl create -f snapshot.yaml
error: error validating "snapshot.yaml": error validating data: [ValidationError(Job.spec.template.spec.volumes[0]): unknown field "path" in io.k8s.api.core.v1.Volume, ValidationError(Job.spec.template.spec.volumes[0]): unknown field "type" in io.k8s.api.core.v1.Volume, ValidationError(Job.spec.template.spec.volumes[1]): unknown field "path" in io.k8s.api.core.v1.Volume, ValidationError(Job.spec.template.spec.volumes[1]): unknown field "type" in io.k8s.api.core.v1.Volume]; if you choose to ignore these errors, turn validation off with --validate=false

根因分析 #

报错明确提示volumes字段下存在不识别的path、type属性，问题出在volumes部分的YAML缩进错误：

• hostPath是Volume资源下的一个字段，值为对象，该对象才包含path、type两个子属性，原配置中path、type的缩进层级和hostPath平级，没有被识别为hostPath的子字段
• name是Volume资源的顶级字段，应该和hostPath平级，原配置中name被错误缩进，和path、type放在了同一层级

Kubernetes API校验时，直接在Volume结构体下查找path、type字段，找不到对应定义就抛出了未知字段的错误。

修复后的正确配置 #

仅需要调整volumes部分的缩进即可，修正后的完整YAML如下：

apiVersion: batch/v1
kind: Job
metadata:
  name: backup
  namespace: management
spec:
  template:
    spec:
      containers:
      - command:
        - /bin/sh
        args:
        - -ec
        - etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key snapshot save /snapshots/backup.db
        image: k8s.gcr.io/etcd-amd64:3.1.12
        name: etcdctl
        env:
        - name: ETCDCTL_API
          value: '3'
        volumeMounts:
        - mountPath: /etc/kubernetes/pki/etcd
          name: etcd-certs
          readOnly: true
        - mountPath: /snapshots
          name: snapshots
      hostNetwork: true
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node-role.kubernetes.io/master
                operator: Exists
      restartPolicy: OnFailure
      tolerations:
      - effect: NoSchedule
        operator: Exists
      volumes:
      - name: etcd-certs
        hostPath:
          path: /etc/kubernetes/pki/etcd
          type: DirectoryOrCreate
      - name: snapshots
        hostPath:
          path: /snapshots
          type: DirectoryOrCreate

补充说明：如果是1.24及以上版本的kubeadm部署集群，控制平面节点的标签为node-role.kubernetes.io/control-plane，如果调度失败可以在nodeAffinity的matchExpressions中补充该标签的匹配规则。

修正后重新执行kubectl create -f snapshot.yaml即可正常创建Job资源。

内容的提问来源于stack exchange，提问作者Johsttin Curahua