Kubernetes：在NodePool级别隔离未声明资源的工作负载至独立节点池

完全可行，不需要修改任何应用的Deployment配置，只需通过NodePool污点配置+集群级Mutating Admission Webhook实现自动引流。以下是具体实现步骤（基于Terraform）：

1. 创建带污点和专属标签的NodePool #

首先用Terraform创建独立NodePool，给节点添加污点阻止普通Pod调度，同时添加专属标签用于精准定位：

resource "google_container_node_pool" "unset_resource_pool" {
  name       = "unset-resource-pool"
  cluster    = google_container_cluster.main.name
  location   = google_container_cluster.main.location
  node_count = 3

  node_config {
    machine_type = "e2-medium"
    # 添加专属标签，用于后续Pod亲和性匹配
    labels = {
      "pool-type" = "unset-resource"
    }
    # 添加污点，默认拒绝所有无对应容忍的Pod
    taints {
      key    = "resource-unset"
      value  = "yes"
      effect = "NO_SCHEDULE"
    }
  }
}

2. 部署Mutating Admission Webhook自动注入调度规则 #

这个Webhook会在Pod创建/更新时自动检查资源声明情况，给未声明/部分声明资源的Pod注入对应的容忍度和节点亲和性，无需修改任何应用配置：

核心逻辑 #

Webhook会校验Pod的每个容器：

• 如果容器未设置resources.requests或resources.limits（任意资源类型，比如CPU、内存）
• 自动注入：
  • 容忍NodePool的污点，允许调度到目标节点
  • 节点亲和性，强制Pod调度到带pool-type=unset-resource标签的NodePool

Terraform部署示例 #

# 创建Webhook服务账号
resource "kubernetes_service_account" "webhook_sa" {
  metadata {
    name      = "resource-mutation-webhook-sa"
    namespace = "kube-system"
  }
}

# 部署Webhook实例（需自行构建或使用符合逻辑的Webhook镜像）
resource "kubernetes_deployment" "resource_mutation_webhook" {
  metadata {
    name      = "resource-mutation-webhook"
    namespace = "kube-system"
  }
  spec {
    replicas = 2
    selector {
      match_labels = {
        app = "resource-mutation-webhook"
      }
    }
    template {
      metadata {
        labels = {
          app = "resource-mutation-webhook"
        }
      }
      spec {
        containers {
          name  = "webhook"
          image = "your-custom-webhook-image:v1" # 替换为你的Webhook镜像
          ports {
            container_port = 443
          }
          volume_mounts {
            name       = "webhook-cert"
            mount_path = "/cert"
            read_only  = true
          }
        }
        volumes {
          name = "webhook-cert"
          secret {
            secret_name = "resource-mutation-webhook-cert" # 提前创建Webhook的SSL证书密钥
          }
        }
        service_account_name = kubernetes_service_account.webhook_sa.metadata[0].name
      }
    }
  }
}

# 创建Webhook服务
resource "kubernetes_service" "webhook_svc" {
  metadata {
    name      = "resource-mutation-webhook-svc"
    namespace = "kube-system"
  }
  spec {
    selector = {
      app = "resource-mutation-webhook"
    }
    ports {
      port        = 443
      target_port = 443
    }
  }
}

# 配置Mutating Webhook规则
resource "kubernetes_mutating_webhook_configuration" "resource_mutation" {
  metadata {
    name = "resource-mutation.webhook.example.com"
  }
  webhook {
    name = "resource-mutation.webhook.example.com"
    client_config {
      service {
        name      = kubernetes_service.webhook_svc.metadata[0].name
        namespace = kubernetes_service.webhook_svc.metadata[0].namespace
        path      = "/mutate"
      }
      ca_bundle = file("./webhook-ca.crt") # 替换为你的Webhook CA证书文件路径
    }
    rules {
      operations = ["CREATE", "UPDATE"]
      api_groups = [""]
      api_versions = ["v1"]
      resources = ["pods"]
    }
    # 可选：Webhook故障时忽略，避免影响集群正常调度
    failure_policy = "Ignore"
  }
}

3. 验证效果 #

部署完成后，创建一个未声明资源的Pod，检查它是否自动被调度到unset-resource-pool节点；同时创建一个带完整资源声明的Pod，确认它不会被调度到该NodePool。

内容的提问来源于stack exchange，提问作者We are Borg