如何让DaemonSet Pod访问StatefulSet创建的Azure Disk PVC?
通过StatefulSet部署了3个app-1 Pod,配置如下:
apiVersion: apps/v1 kind: StatefulSet metadata: name: app-1 spec: replicas: 3 selector: matchLabels: app: app-1 serviceName: "app-1" template: metadata: labels: app: app-1 spec: containers: - name: app-1 image: registry.k8s.io/nginx-slim:0.8 resources: requests: cpu: 100m memory: 128Mi limits: cpu: 250m memory: 256Mi ports: - containerPort: 4567 volumeMounts: - name: app-1-state-volume-claim mountPath: /app1Data - name: app-2-data-volume-claim mountPath: /app2Data volumeClaimTemplates: - metadata: name: app-1-state-volume-claim spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-csi-premium" resources: requests: storage: 1Gi - metadata: name: app-2-data-volume-claim spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-csi-premium" resources: requests: storage: 1Gi
app-1的状态存储在app-1-state-volume-claim PVC中,同时会在app-2-data-volume-claim PVC模板生成的存储中写入供app-2使用的数据。
尝试通过DaemonSet部署app-2 Pod访问该存储,DaemonSet配置如下:
apiVersion: apps/v1 kind: DaemonSet metadata: name: app-2 spec: selector: matchLabels: name: app-2 template: metadata: labels: name: app-2 spec: containers: - name: app-2 image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2 resources: limits: memory: 200Mi requests: cpu: 100m memory: 200Mi volumeMounts: - name: app2Data mountPath: /app2Data volumes: - name: app2Data persistentVolumeClaim: claimName: app-2-data-volume-claim
部署时出现错误:
persistentvolumeclaim "app-2-data-volume-claim" not found
由于app-1的限制,无法使用Azure文件共享,请问该如何解决此问题?
1. 先理清PVC命名规则
StatefulSet的volumeClaimTemplates会生成带Pod序号后缀的PVC,格式为<模板名称>-<StatefulSet名称>-<Pod序号>。因此3个app-1 Pod对应的app-2-data-volume-claim系列PVC实际名称为:
app-2-data-volume-claim-app-1-0app-2-data-volume-claim-app-1-1app-2-data-volume-claim-app-1-2
直接引用app-2-data-volume-claim肯定找不到,这是错误的核心原因。
2. 方案一:将app-2作为sidecar嵌入app-1的StatefulSet
这是最简单的解决方式,无需额外处理PVC调度问题:把app-2容器直接添加到app-1的Pod模板中,和app-1共享同一份PVC挂载,直接访问/app2Data路径。修改后的app-1 StatefulSet配置如下:
apiVersion: apps/v1 kind: StatefulSet metadata: name: app-1 spec: replicas: 3 selector: matchLabels: app: app-1 serviceName: "app-1" template: metadata: labels: app: app-1 spec: containers: - name: app-1 image: registry.k8s.io/nginx-slim:0.8 resources: requests: cpu: 100m memory: 128Mi limits: cpu: 250m memory: 256Mi ports: - containerPort: 4567 volumeMounts: - name: app-1-state-volume-claim mountPath: /app1Data - name: app-2-data-volume-claim mountPath: /app2Data # 添加app-2作为sidecar容器 - name: app-2 image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2 resources: limits: memory: 200Mi requests: cpu: 100m memory: 200Mi volumeMounts: - name: app-2-data-volume-claim mountPath: /app2Data volumeClaimTemplates: - metadata: name: app-1-state-volume-claim spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-csi-premium" resources: requests: storage: 1Gi - metadata: name: app-2-data-volume-claim spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-csi-premium" resources: requests: storage: 1Gi
3. 方案二:节点绑定式DaemonSet部署
如果必须保留DaemonSet独立部署的模式,可通过节点标签绑定实现每个节点的DaemonSet Pod挂载对应节点上的app-1 PVC:
- 给每个PVC打上对应节点的标签:
for pod in app-1-0 app-1-1 app-1-2; do node=$(kubectl get pod $pod -o jsonpath='{.spec.nodeName}') pvc=$(kubectl get pod $pod -o jsonpath='{.spec.volumes[?(@.name=="app-2-data-volume-claim")].persistentVolumeClaim.claimName}') kubectl label pvc $pvc node=$node done - 为每个节点创建专属DaemonSet,只调度到目标节点并挂载对应PVC(以节点
node-xx为例):apiVersion: apps/v1 kind: DaemonSet metadata: name: app-2-node-xx spec: selector: matchLabels: name: app-2-node-xx template: metadata: labels: name: app-2-node-xx spec: nodeSelector: kubernetes.io/hostname: node-xx containers: - name: app-2 image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2 resources: limits: memory: 200Mi requests: cpu: 100m memory: 200Mi volumeMounts: - name: app2Data mountPath: /app2Data volumes: - name: app2Data persistentVolumeClaim: claimName: app-2-data-volume-claim-app-1-0
重复此步骤为每个运行app-1的节点创建对应DaemonSet即可。
4. 方案三:将app-2改为StatefulSet与app-1一一对应
如果app-2不需要每个节点一个Pod,而是和app-1 Pod一一对应,可将app-2改为StatefulSet,直接引用对应序号的PVC:
apiVersion: apps/v1 kind: StatefulSet metadata: name: app-2 spec: replicas: 3 selector: matchLabels: name: app-2 serviceName: "app-2" template: metadata: labels: name: app-2 spec: # 亲和性配置,确保app-2 Pod和对应app-1 Pod调度到同一节点 affinity: podAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - app-1 topologyKey: "kubernetes.io/hostname" containers: - name: app-2 image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2 resources: limits: memory: 200Mi requests: cpu: 100m memory: 200Mi volumeMounts: - name: app2Data mountPath: /app2Data volumes: - name: app2Data persistentVolumeClaim: claimName: app-2-data-volume-claim-app-1-0 # 对应app-1-0的PVC
注意:此方案需要手动为每个app-2 Pod指定对应序号的PVC,或通过自定义控制器实现自动绑定。
内容的提问来源于stack exchange,提问作者Atul Takekar

