K8s v1.28启用SidecarContainers后Pod仍Pending问题求助

问题描述 #

使用Kubernetes v1.28版本尝试启用SidecarContainers alpha特性，已在API Server配置文件/etc/kubernetes/manifests/kube-apiserver.yaml中添加--feature-gates=SidecarContainers=true，但部署的Pod始终处于Pending状态。

执行kubectl describe po sidecar-demo后，事件中出现如下警告：

Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  40s   default-scheduler  0/2 nodes are available: 2 Pod has a restartable init container and the SidecarContainers feature is disabled. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling..

相关配置文件 #

Deployment配置

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: sidecar-demo
  name: sidecar-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sidecar-demo
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: sidecar-demo
    spec:
      containers:
      - image: alpine:latest
        name: main
        command: ['sh', '-c', 'while true; do echo "logging" >> /opt/logs.txt; sleep 1; done']
        volumeMounts:
          - name: data
            mountPath: /opt
      initContainers:
      - image: alpine:latest
        name: sidecar
        restartPolicy: Always
        command: ['sh', '-c', 'tail -F /opt/logs.txt']
        volumeMounts:
          - name: data
            mountPath: /opt
      volumes:
        - name: data
          emptyDir: {}

API Server配置（关键部分）

apiVersion: v1
kind: Pod
metadata:
  namespace: kube-system
  name: kube-apiserver
spec:
  containers:
  - command:
    - kube-apiserver
    # 省略其他参数
    - --feature-gates=SidecarContainers=true
    image: registry.k8s.io/kube-apiserver:v1.28.1

问题根源 #

SidecarContainers作为alpha特性，仅在API Server开启特性门控是不够的，必须同时在调度器（Scheduler）和所有节点的Kubelet上开启该特性门控。当前仅API Server启用了特性，调度器未识别到特性开启，因此判定Pod的可重启init容器配置不合法，拒绝调度。

解决步骤 #

1. 启用调度器的SidecarContainers特性 #

编辑调度器配置文件/etc/kubernetes/manifests/kube-scheduler.yaml，在command字段中添加--feature-gates=SidecarContainers=true：

apiVersion: v1
kind: Pod
metadata:
  namespace: kube-system
  name: kube-scheduler
spec:
  containers:
  - command:
    - kube-scheduler
    # 省略其他参数
    - --feature-gates=SidecarContainers=true
    image: registry.k8s.io/kube-scheduler:v1.28.1

保存后，kubelet会自动重启调度器Pod，等待其进入Running状态。

2. 启用所有节点Kubelet的SidecarContainers特性 #

在每个节点上执行以下操作：

• 编辑Kubelet配置文件/var/lib/kubelet/config.yaml，添加或修改featureGates字段：

  featureGates:
    SidecarContainers: true
  

• 重启Kubelet服务：

  systemctl restart kubelet
  

3. 验证特性启用状态 #

• 检查API Server：

  kubectl get pods -n kube-system kube-apiserver-<节点名称> -o yaml | grep -A5 "feature-gates"
  

• 检查调度器：

  kubectl get pods -n kube-system kube-scheduler-<节点名称> -o yaml | grep -A5 "feature-gates"
  

• 检查节点Kubelet：

  kubectl get node <节点名称> -o yaml | grep -A5 "featureGates"
  

确保所有组件的特性门控均已设置为SidecarContainers=true。

4. 重新部署Pod #

删除现有Deployment并重新应用配置：

kubectl delete deployment sidecar-demo
kubectl apply -f <你的Deployment文件路径>

此时Pod应能正常调度并进入Running状态。

内容的提问来源于stack exchange，提问作者chadmcrowell