使用Terraform+CSV在Azure同一VNet创建多子网遇重复键错误求助

错误现象 #

执行terraform apply时触发重复键错误：

azureuser@aztfserver:~/az_csv_tasks_new$ sudo terraform apply  -auto-approve
╷
│ Error: Duplicate object key
│
│   on rg_vnets_subnets.tf line 24, in resource "azurerm_resource_group" "rg":
│   24:   for_each = { for v in local.vnet_config : v.resource_group_name => v }
│     ├────────────────
│     │ v.resource_group_name is "rg-ScriptTest-wus-002"
│
│ Two different items produced the key "rg-ScriptTest-wus-002" in this 'for' expression. If duplicates are expected, use the  
│ ellipsis (...) after the value expression to enable grouping by key.

添加省略号后仍报错，提示信息一致。

配置文件内容 #

rg_vnets_subnets.csv #

resource_group_name,location,vnet_name,vnet_address_space,subnet_name,subnet_address_space
rg-ScriptTest-eus-001,East US,vnet-ScriptTest-eus-001,10.0.0.0/16,subnet1,10.0.1.0/26
rg-ScriptTest-wus-002,West US,vnet-ScriptTest-wus-002,11.0.0.0/16,subnet2,11.0.1.0/26
rg-ScriptTest-wus-002,West US,vnet-ScriptTest-wus-002,11.0.0.0/16,subnet3,11.0.2.0/26

main.tf #

locals {
  vnet_config = csvdecode(file("${path.module}/rg_vnets_subnets.csv"))
}

resource "azurerm_resource_group" "rg" {
  for_each = { for v in local.vnet_config : v.resource_group_name => v }
  name     = each.value.resource_group_name
  location = each.value.location
}

resource "azurerm_virtual_network" "vnet" {
  for_each            = { for v in local.vnet_config : v.vnet_name => v }
  name                = each.value.vnet_name
  location            = each.value.location
  resource_group_name = each.value.resource_group_name
  address_space       = [each.value.vnet_address_space]
  depends_on          = [azurerm_resource_group.rg]
}

resource "azurerm_subnet" "subnet" {
  for_each            = { for v in local.vnet_config : "${v.vnet_name}-${v.subnet_name}" => v }
  name                = each.value.subnet_name
  resource_group_name = each.value.resource_group_name
  virtual_network_name = each.value.vnet_name
 address_prefixes     = [each.value.subnet_address_space]
 depends_on          = [azurerm_virtual_network.vnet]
}

问题原因 #

CSV中存在同一资源组/虚拟网络对应多个子网的行，导致用resource_group_name/vnet_name作为for_each的键时出现重复值，Terraform不允许映射键重复。你尝试添加省略号但未正确使用分组语法，所以错误依然存在。

解决方法 #

需要对资源组和虚拟网络的配置进行去重，确保每个键唯一，子网部分的组合键已经是唯一的，无需修改。

修改后的main.tf（分组写法） #

locals {
  vnet_config = csvdecode(file("${path.module}/rg_vnets_subnets.csv"))
  
  # 按资源组名分组，取每组第一个元素（同一RG配置一致）
  unique_rgs = { 
    for rg_name, rg_items in groupby(local.vnet_config, "resource_group_name") : 
    rg_name => rg_items[0] 
  }
  
  # 按虚拟网络名分组，取每组第一个元素（同一VNet配置一致）
  unique_vnets = { 
    for vnet_name, vnet_items in groupby(local.vnet_config, "vnet_name") : 
    vnet_name => vnet_items[0] 
  }
}

resource "azurerm_resource_group" "rg" {
  for_each = local.unique_rgs
  name     = each.value.resource_group_name
  location = each.value.location
}

resource "azurerm_virtual_network" "vnet" {
  for_each            = local.unique_vnets
  name                = each.value.vnet_name
  location            = each.value.location
  resource_group_name = each.value.resource_group_name
  address_space       = [each.value.vnet_address_space]
  depends_on          = [azurerm_resource_group.rg]
}

resource "azurerm_subnet" "subnet" {
  for_each            = { for v in local.vnet_config : "${v.vnet_name}-${v.subnet_name}" => v }
  name                = each.value.subnet_name
  resource_group_name = each.value.resource_group_name
  virtual_network_name = each.value.vnet_name
 address_prefixes     = [each.value.subnet_address_space]
 depends_on          = [azurerm_virtual_network.vnet]
}

简化去重写法 #

如果确认同一资源组/虚拟网络的配置完全一致，也可以用distinct函数直接去重：

resource "azurerm_resource_group" "rg" {
  for_each = { for v in distinct(local.vnet_config) : v.resource_group_name => v }
  name     = each.value.resource_group_name
  location = each.value.location
}

resource "azurerm_virtual_network" "vnet" {
  for_each            = { for v in distinct(local.vnet_config) : v.vnet_name => v }
  name                = each.value.vnet_name
  location            = each.value.location
  resource_group_name = each.value.resource_group_name
  address_space       = [each.value.vnet_address_space]
  depends_on          = [azurerm_resource_group.rg]
}

CSV作为变量的建议 #

可以将CSV文件路径定义为Terraform变量，提升配置灵活性，方便不同环境切换配置文件：

1. 创建variables.tf：

variable "csv_config_path" {
  type        = string
  description = "Path to the CSV file containing VNet and Subnet configurations"
  default     = "rg_vnets_subnets.csv"
}

2. 修改main.tf中的local块：

locals {
  vnet_config = csvdecode(file(var.csv_config_path))
}

执行terraform apply时，可通过-var csv_config_path="other_config.csv"指定不同的CSV文件。

内容的提问来源于stack exchange，提问作者sharan kumar