Terraform代码复用：如何通过循环模板消除冗余？

一、Terraform原生实现：用for_each循环复用代码 #

你可以通过定义参数化变量列表，配合for_each遍历创建重复资源，完全不需要多个重复文件。具体步骤如下：

1. 在jobs模块中定义参数化变量 #

在jobs模块的variables.tf里，新增一个描述所有Glue Job配置的变量：

variable "glue_jobs" {
  type = list(object({
    name                  = string
    script_key            = string
    script_source_path    = string
    raw_folder_key        = string
    schedule              = string
    max_concurrent_runs   = number
    max_retries           = number
    number_of_workers     = number
    worker_type           = string
    timeout               = number
    glue_version          = string
  }))
  description = "所有需要创建的Glue Job配置列表"
}

# 保留原有的其他变量
variable "environment" {}
variable "region" {}
variable "name" {}
variable "bucket_id" {}
variable "bucket_name" {}
variable "glue_role_arn" {}
variable "databricks_jdbc_connection_name" {}

2. 改写资源为循环创建模式 #

把jobs模块内的所有资源，用for_each遍历var.glue_jobs变量，动态生成实例：

S3脚本文件

resource "aws_s3_object" "job_script" {
  for_each = { for job in var.glue_jobs : job.name => job }
  bucket   = var.bucket_id
  key      = each.value.script_key
  source   = each.value.script_source_path
}

S3原始文件夹

resource "aws_s3_object" "raw_folder" {
  for_each = { for job in var.glue_jobs : job.name => job }
  bucket   = var.bucket_id
  key      = each.value.raw_folder_key
}

Glue Job

resource "aws_glue_job" "main" {
  for_each = { for job in var.glue_jobs : job.name => job }
  name     = each.value.name
  role_arn = var.glue_role_arn
  execution_class = "STANDARD"

  execution_property {
    max_concurrent_runs = each.value.max_concurrent_runs
  }

  command {
    name            = "glueetl"
    script_location = "s3://${var.bucket_name}/${aws_s3_object.job_script[each.key].key}"
  }

  connections = [var.databricks_jdbc_connection_name]

  max_retries        = each.value.max_retries
  number_of_workers  = each.value.number_of_workers
  worker_type        = each.value.worker_type
  timeout            = each.value.timeout
  glue_version       = each.value.glue_version
}

Glue调度触发器

resource "aws_glue_trigger" "scheduled" {
  for_each = { for job in var.glue_jobs : job.name => job }
  name     = each.value.name
  schedule = each.value.schedule
  type     = "SCHEDULED"

  actions {
    job_name = aws_glue_job.main[each.key].name
  }
}

3. 调用模块时传入参数列表 #

在根模块调用jobs时，直接传入所有Job的配置参数即可：

module "jobs" {
  source = "./jobs"

  environment                     = var.environment
  region                          = var.region
  name                            = var.name
  bucket_id                       = module.commons.bucket_id
  bucket_name                     = module.commons.bucket_name
  glue_role_arn                   = var.glue_role_arn
  databricks_jdbc_connection_name = var.databricks_jdbc_connection_name

  # 这里可以添加任意多个Job配置
  glue_jobs = [
    {
      name                  = "test_samples"
      script_key            = "scripts/test_samples.sql"
      script_source_path    = "../src/main/resources/test_samples.sql"
      raw_folder_key        = "raw/test_samples/"
      schedule              = "cron(00 02 ? * SUN *)"
      max_concurrent_runs   = 1
      max_retries           = 0
      number_of_workers     = 2
      worker_type           = "G.1X"
      timeout               = 10
      glue_version          = "3.0"
    },
    {
      name                  = "daily_etl_job"
      script_key            = "scripts/daily_etl.sql"
      script_source_path    = "../src/main/resources/daily_etl.sql"
      raw_folder_key        = "raw/daily_etl/"
      schedule              = "cron(00 01 ? * * *)"
      max_concurrent_runs   = 1
      max_retries           = 1
      number_of_workers     = 4
      worker_type           = "G.2X"
      timeout               = 15
      glue_version          = "3.0"
    }
  ]
}

二、Terragrunt在这个场景中的作用 #

Terragrunt主要用于多环境/多模块的配置复用与管理，它不能替代Terraform的for_each实现资源级别的代码复用，但可以帮你减少模块调用时的重复配置：

典型用法示例 #

假设你有dev、prod多个环境，可通过Terragrunt的配置继承来共享公共参数：

1. 项目结构：

├── dev
│   └── terragrunt.hcl
├── prod
│   └── terragrunt.hcl
├── modules
│   └── jobs
└── terragrunt.hcl  # 公共配置

2. 公共配置文件terragrunt.hcl：

locals {
  common_config = {
    region                          = "us-east-1"
    commons_bucket_id               = "my-shared-bucket"
    commons_bucket_name             = "my-shared-bucket"
    glue_role_arn                   = "arn:aws:iam::123456789012:role/glue-exec-role"
    databricks_jdbc_connection_name = "databricks-jdbc-conn"
  }
}

3. 环境配置文件（如dev/terragrunt.hcl）：

terraform {
  source = "../modules/jobs"
}

include {
  path = find_in_parent_folders()
}

inputs = {
  environment = "dev"
  name = "dev-jobs"
  bucket_id = local.common_config.commons_bucket_id
  bucket_name = local.common_config.commons_bucket_name
  glue_role_arn = local.common_config.glue_role_arn
  databricks_jdbc_connection_name = local.common_config.databricks_jdbc_connection_name

  glue_jobs = [
    {
      name                  = "test_samples-dev"
      script_key            = "scripts/test_samples.sql"
      script_source_path    = "../../src/main/resources/test_samples.sql"
      raw_folder_key        = "raw/test_samples-dev/"
      schedule              = "cron(00 02 ? * SUN *)"
      max_concurrent_runs   = 1
      max_retries           = 0
      number_of_workers     = 2
      worker_type           = "G.1X"
      timeout               = 10
      glue_version          = "3.0"
    }
  ]
}

这样你不用在每个环境的模块调用里重复写相同的公共参数，Terragrunt会自动继承公共配置。

内容的提问来源于stack exchange，提问作者Meyer Cohen