如何在EKS集群配置MostAllocated及实现Pod紧凑调度？

一、EKS中配置MostAllocated评分策略配合Descheduler的HighNodeUtilization #

EKS的默认kube-scheduler为托管组件，无法直接修改配置，需部署自定义调度器来启用MostAllocated评分策略，具体步骤如下：

1. 编写调度器配置文件
   创建kube-scheduler-config.yaml，指定NodeResourcesFit插件采用MostAllocated评分策略（以CPU、内存为例，可按需调整资源类型与权重）：

   apiVersion: kubescheduler.config.k8s.io/v1beta3
   kind: KubeSchedulerConfiguration
   profiles:
   - name: default
     pluginConfig:
     - name: NodeResourcesFit
       args:
         scoringStrategy:
           type: MostAllocated
           resources:
           - name: cpu
             weight: 1
           - name: memory
             weight: 1
   leaderElection:
     leaderElect: true
     resourceName: custom-scheduler
     resourceNamespace: kube-system
   

2. 创建ConfigMap存储配置

   kubectl create configmap custom-scheduler-config -n kube-system --from-file=kube-scheduler-config.yaml
   

3. 部署自定义调度器
   编写custom-scheduler-deployment.yaml，使用与EKS集群版本匹配的kube-scheduler镜像（示例为v1.28版本），并绑定必要权限：

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: custom-scheduler
     namespace: kube-system
     labels:
       component: scheduler
       tier: control-plane
   spec:
     replicas: 1
     selector:
       matchLabels:
         component: scheduler
         tier: control-plane
     template:
       metadata:
         labels:
           component: scheduler
           tier: control-plane
       spec:
         serviceAccountName: custom-scheduler
         containers:
         - name: custom-scheduler
           image: public.ecr.aws/eks-distro/kubernetes/kube-scheduler:v1.28.2-eks-1-28-5
           args:
           - --config=/etc/kubernetes/scheduler-config/kube-scheduler-config.yaml
           - --leader-elect=true
           volumeMounts:
           - name: scheduler-config
             mountPath: /etc/kubernetes/scheduler-config
         volumes:
         - name: scheduler-config
           configMap:
             name: custom-scheduler-config
   ---
   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: custom-scheduler
     namespace: kube-system
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: custom-scheduler-as-kube-scheduler
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: system:kube-scheduler
   subjects:
   - kind: ServiceAccount
     name: custom-scheduler
     namespace: kube-system
   

   部署该资源：

   kubectl apply -f custom-scheduler-deployment.yaml
   

4. 配置Descheduler的HighNodeUtilization策略
   确保Descheduler配置中启用该策略，示例descheduler-policy.yaml：

   apiVersion: descheduler/v1alpha2
   kind: DeschedulerPolicy
   strategies:
     HighNodeUtilization:
       enabled: true
       params:
         nodeResourceUtilizationThresholds:
           thresholds:
             cpu: 80
             memory: 80
           targetThresholds:
             cpu: 90
             memory: 90
   

   部署Descheduler时挂载该配置，并为目标Pod指定自定义调度器（在Pod模板中添加schedulerName: custom-scheduler）。

二、不使用Descheduler实现Pod紧凑部署 #

可通过以下两种方案实现，无需依赖Descheduler：

方案1：自定义调度器+Cluster Autoscaler #

• 按上述步骤部署启用MostAllocated策略的自定义调度器，让新Pod优先调度到资源利用率更高的节点。
• 配置Cluster Autoscaler，设置--scale-down-unneeded-time=10m（可按需调整），让空节点或低负载节点快速缩容，同时确保Cluster Autoscaler拥有节点删除权限，且节点组配置了正确的缩容规则。

方案2：使用Karpenter实现自动紧凑部署 #

Karpenter是AWS推荐的节点自动扩缩容工具，自带Pod consolidation功能：

1. 将Karpenter部署至EKS集群。
2. 创建Provisioner配置并启用consolidation：

   apiVersion: karpenter.sh/v1alpha5
   kind: Provisioner
   metadata:
     name: default
   spec:
     consolidation:
       enabled: true
     requirements:
       - key: kubernetes.io/arch
         operator: In
         values: ["amd64"]
       - key: kubernetes.io/os
         operator: In
         values: ["linux"]
     ttlSecondsAfterEmpty: 30
   

   Karpenter会自动监测节点负载，将低负载节点上的Pod重新调度至其他节点，随后删除空节点，实现Pod紧凑部署。

内容的提问来源于stack exchange，提问作者overflow_warrior_27