如何基于Terraform对象集合与分区数批量创建资源？

我定义了如下Terraform变量：

variable "offset_counter" {
  description = "Represents topic and consumer group properties"
  type = set(object({
    topic_name              = string
    topic_partitions_number = number
    consumer_group_name     = string
    max_offset_lag          = number
  }))
}

希望基于topic_name和topic_partitions_number批量创建资源——比如2个Topic（topic A和topic B），topic A有20个分区、topic B有50个分区，最终生成70个资源。

变量示例取值如下：

offset_counter = [
  {
    topic_name              = "topic_A"
    topic_partitions_number = 2
    consumer_group_name     = "my_consumer_group_A"
    max_offset_lag          = 200
  },
  {
    topic_name              = "topic_B"
    topic_partitions_number = 3
    consumer_group_name     = "my_consumer_group_B"
    max_offset_lag          = 50
  }
]

期望生成的资源形式如下：

resource "aws_cloudwatch_metric_alarm" "offset-lag-too-high" {
  alarm_name = "Alarm for topic topic_A on partition 1: offset lag is more then 200. Consumer group: my_consumer_group_A"
  # 其他告警配置
}

resource "aws_cloudwatch_metric_alarm" "offset-lag-too-high" {
  alarm_name = "Alarm for topic topic_A on partition 2: offset lag is more then 200. Consumer group: my_consumer_group_A"
  # 其他告警配置
}

resource "aws_cloudwatch_metric_alarm" "offset-lag-too-high" {
  alarm_name = "Alarm for topic topic_B on partition 1: offset lag is more then 50. Consumer group: my_consumer_group_B"
  # 其他告警配置
}

请问是否可以实现？

完全可以实现，核心是用Terraform的flatten函数结合range，把每个Topic的分区展开为独立元素，再通过for_each遍历生成资源。

步骤1：构造遍历用的集合 #

先通过flatten和for表达式，将每个Topic的分区拆分为单独条目：

locals {
  topic_partitions = flatten([
    for topic in var.offset_counter : [
      for partition in range(1, topic.topic_partitions_number + 1) : {
        topic_name          = topic.topic_name
        partition_number    = partition
        consumer_group_name = topic.consumer_group_name
        max_offset_lag      = topic.max_offset_lag
      }
    ]
  ])
}

这个局部变量会把示例变量转换为5个独立条目（topic_A的2个分区 + topic_B的3个分区）。

步骤2：批量生成告警资源 #

使用for_each遍历local.topic_partitions，为每个分区创建单独的CloudWatch告警：

resource "aws_cloudwatch_metric_alarm" "offset-lag-too-high" {
  for_each = { for tp in local.topic_partitions : "${tp.topic_name}-partition-${tp.partition_number}" => tp }

  alarm_name = "Alarm for topic ${each.value.topic_name} on partition ${each.value.partition_number}: offset lag is more than ${each.value.max_offset_lag}. Consumer group: ${each.value.consumer_group_name}"
  # 补充其他必要的告警配置，示例如下：
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = "2"
  threshold           = each.value.max_offset_lag
  metric_name         = "OffsetLag"
  namespace           = "AWS/Kafka" # 根据实际监控服务调整
  statistic           = "Average"
  period              = 60
  alarm_description   = "Triggered when offset lag exceeds ${each.value.max_offset_lag} for ${each.value.topic_name} partition ${each.value.partition_number}"
}

关键说明 #

• for_each使用复合键${tp.topic_name}-partition-${tp.partition_number}确保每个资源实例唯一，避免Terraform报错。
• 通过each.value可访问每个分区对应的Topic名称、分区号、消费组和阈值等参数，动态生成告警名称和配置。

内容的提问来源于stack exchange，提问作者Vitalii