如何用Terraform变量创建多个同类型复杂K8s节点池资源

针对多集群节点池差异化配置的复用需求，推荐使用**list(object)变量定义+dynamic块动态生成**的方案，完全支持任意数量的节点池配置，无需单独为每个节点池创建变量或资源。

1. 定义节点池变量（variables.tf） #

用list(object)类型完整定义节点池的所有配置字段，嵌套结构与GKE Provider的node_pool参数完全对齐，可选字段用optional()标记以兼容差异化配置：

variable "node_pools" {
  type = list(object({
    initial_node_count    = number
    max_pods_per_node     = number
    name                  = string
    node_count            = number
    node_locations        = list(string)
    version               = string
    management = object({
      auto_repair  = bool
      auto_upgrade = bool
    })
    node_config = object({
      disk_size_gb      = number
      disk_type         = string
      guest_accelerator = list(object({
        count  = number
        type   = string
      }))
      image_type        = string
      labels            = map(string)
      local_ssd_count   = number
      machine_type      = string
      metadata          = map(string)
      oauth_scopes      = list(string)
      preemptible       = bool
      service_account   = string
      spot              = bool
      tags              = list(string)
      taint = list(object({
        effect = string
        key    = string
        value  = string
      }))
      shielded_instance_config = object({
        enable_integrity_monitoring = bool
        enable_secure_boot          = bool
      })
      workload_metadata_config = optional(object({
        mode = string
      }))
    })
    upgrade_settings = object({
      max_surge       = number
      max_unavailable = number
    })
    instance_group_urls = optional(list(string), [])
  }))
  description = "GKE集群的节点池列表，支持任意数量的差异化配置"
}

2. 在GKE集群资源中动态生成节点池（main.tf） #

使用Terraform的dynamic块遍历var.node_pools列表，自动生成每个节点池的配置块：

resource "google_container_cluster" "primary" {
  name     = "your-cluster-name"
  location = "us-central1"
  # 集群基础配置（如网络、IP范围等）...

  # 动态生成节点池
  dynamic "node_pool" {
    for_each = var.node_pools
    content {
      initial_node_count    = node_pool.value.initial_node_count
      max_pods_per_node     = node_pool.value.max_pods_per_node
      name                  = node_pool.value.name
      node_count            = node_pool.value.node_count
      node_locations        = node_pool.value.node_locations
      version               = node_pool.value.version
      instance_group_urls   = node_pool.value.instance_group_urls

      management {
        auto_repair  = node_pool.value.management.auto_repair
        auto_upgrade = node_pool.value.management.auto_upgrade
      }

      node_config {
        disk_size_gb      = node_pool.value.node_config.disk_size_gb
        disk_type         = node_pool.value.node_config.disk_type
        guest_accelerator = node_pool.value.node_config.guest_accelerator
        image_type        = node_pool.value.node_config.image_type
        labels            = node_pool.value.node_config.labels
        local_ssd_count   = node_pool.value.node_config.local_ssd_count
        machine_type      = node_pool.value.node_config.machine_type
        metadata          = node_pool.value.node_config.metadata
        oauth_scopes      = node_pool.value.node_config.oauth_scopes
        preemptible       = node_pool.value.node_config.preemptible
        service_account   = node_pool.value.node_config.service_account
        spot              = node_pool.value.node_config.spot
        tags              = node_pool.value.node_config.tags
        taint             = node_pool.value.node_config.taint

        shielded_instance_config {
          enable_integrity_monitoring = node_pool.value.node_config.shielded_instance_config.enable_integrity_monitoring
          enable_secure_boot          = node_pool.value.node_config.shielded_instance_config.enable_secure_boot
        }

        # 处理可选的workload_metadata_config字段，仅当配置存在时生成
        dynamic "workload_metadata_config" {
          for_each = node_pool.value.node_config.workload_metadata_config != null ? [node_pool.value.node_config.workload_metadata_config] : []
          content {
            mode = workload_metadata_config.value.mode
          }
        }
      }

      upgrade_settings {
        max_surge       = node_pool.value.upgrade_settings.max_surge
        max_unavailable = node_pool.value.upgrade_settings.max_unavailable
      }
    }
  }
}

3. 传入节点池配置（terraform.tfvars） #

将现有节点池的配置直接转换为变量值，示例对应你提供的两个节点池：

node_pools = [
  {
    initial_node_count    = 12
    max_pods_per_node     = 16
    name                  = "test-pool"
    node_count            = 12
    node_locations        = ["us-central1-b", "us-central1-c", "us-central1-f"]
    version               = "1.21.14-gke.700"
    management = {
      auto_repair  = true
      auto_upgrade = true
    }
    node_config = {
      disk_size_gb      = 50
      disk_type         = "pd-standard"
      guest_accelerator = []
      image_type        = "COS_CONTAINERD"
      labels            = {"integrationtestnode" = "true"}
      local_ssd_count   = 0
      machine_type      = "n1-standard-2"
      metadata          = {"disable-legacy-endpoints" = "true"}
      oauth_scopes      = ["https://www.googleapis.com/auth/cloud-platform"]
      preemptible       = false
      service_account   = "svcs-dev@megacorp-dev-project.iam.gserviceaccount.com"
      spot              = false
      tags              = []
      taint = [
        {
          effect = "NO_SCHEDULE"
          key    = "integrationtest"
          value  = "true"
        }
      ]
      shielded_instance_config = {
        enable_integrity_monitoring = true
        enable_secure_boot          = true
      }
    }
    upgrade_settings = {
      max_surge       = 1
      max_unavailable = 0
    }
  },
  {
    initial_node_count    = 1
    max_pods_per_node     = 110
    name                  = "promop-n2s8"
    node_count            = 1
    node_locations        = ["us-central1-b", "us-central1-c", "us-central1-f"]
    version               = "1.21.13-gke.900"
    management = {
      auto_repair  = true
      auto_upgrade = true
    }
    node_config = {
      disk_size_gb      = 100
      disk_type         = "pd-standard"
      guest_accelerator = []
      image_type        = "COS_CONTAINERD"
      labels            = {"megacorp.reserved" = "promop-dev"}
      local_ssd_count   = 0
      machine_type      = "n2-standard-8"
      metadata          = {"disable-legacy-endpoints" = "true"}
      oauth_scopes      = ["https://www.googleapis.com/auth/cloud-platform"]
      preemptible       = false
      service_account   = "svcs-dev@megacorp-dev-project.iam.gserviceaccount.com"
      spot              = false
      tags              = []
      taint = [
        {
          effect = "NO_SCHEDULE"
          key    = "app"
          value  = "prometheus-operator-dev"
        }
      ]
      shielded_instance_config = {
        enable_integrity_monitoring = true
        enable_secure_boot          = false
      }
      workload_metadata_config = {
        mode = "GKE_METADATA"
      }
    }
    upgrade_settings = {
      max_surge       = 2
      max_unavailable = 0
    }
  }
]

方案优势 #

• 支持任意数量的节点池，无需重复编写资源块
• 配置结构与GKE Provider完全匹配，直接复用现有节点池配置
• 通过optional()和dynamic块处理差异化字段，兼容性强
• 多集群场景下，仅需为每个集群传入不同的node_pools列表即可实现复用

内容的提问来源于stack exchange，提问作者TeeTee