调整AWS VPC可用区数量时遇InvalidSubnet.Conflict错误排查

错误信息 #

InvalidSubnet.Conflict: The CIDR 'xx.xx.x.xxx/xx' conflicts with another subnet
status code: 400

相关Terraform配置 #

locals {
  vpc_cidr_start             = "192.168"
  vpc_cidr_size              = var.vpc_cidr_size
  vpc_cidr                   = "${local.vpc_cidr_start}.0.0/${local.vpc_cidr_size}"
  cidr_power                 = 32 - var.vpc_cidr_size
  default_subnet_size_per_az = 27

  public_subnet_ips_num  = (var.use_only_public_subnet ? pow(2, 32 - local.vpc_cidr_size) : pow(2, 32 - local.default_subnet_size_per_az) * length(var.availability_zones))
  private_subnet_ips_num = var.use_only_public_subnet ? 0 : pow(2, 32 - local.vpc_cidr_size) - local.public_subnet_ips_num
  ips_per_private_subnet = format("%b", floor(local.private_subnet_ips_num / length(var.availability_zones)))
  ips_per_public_subnet  = format("%b", floor(local.public_subnet_ips_num / length(var.availability_zones)))
  private_subnet_cidr_size = tolist([
    for i in range(4, length(local.ips_per_private_subnet)) : (32 - local.vpc_cidr_size - i)
    if substr(strrev(local.ips_per_private_subnet), i, 1) == "1"
  ])
  public_subnet_cidr_size = tolist([
    for i in range(4, length(local.ips_per_public_subnet)) : (32 - local.vpc_cidr_size - i)
    if substr(strrev(local.ips_per_public_subnet), i, 1) == "1"
  ])

  subnets_by_az = concat(
    flatten([
      for az in var.availability_zones :
      [
        tolist([
          for s in local.private_subnet_cidr_size : {
            availability_zone = az
            public            = false
            size              = tonumber(s)
          }
        ]),
        tolist([
          for s in local.public_subnet_cidr_size : {
            availability_zone = az
            public            = true
            size              = tonumber(s)
          }
        ])
      ]
    ])
  )

  subnets_by_size    = { for s in local.subnets_by_az : format("%03d", s.size) => s... }
  sorted_subnet_keys = sort(keys(local.subnets_by_size))
  sorted_subnets = flatten([
    for s in local.sorted_subnet_keys :
    local.subnets_by_size[s]
  ])
  sorted_subnet_sizes = flatten([
    for s in local.sorted_subnet_keys :
    local.subnets_by_size[s][*].size
  ])

  subnet_cidrs = length(local.sorted_subnet_sizes) > 0 && local.sorted_subnet_sizes[0] == 0 ? [
    local.vpc_cidr
  ] : cidrsubnets(local.vpc_cidr, local.sorted_subnet_sizes...)

  subnets = flatten([
    for i, subnet in local.sorted_subnets :
    [
      {
        availability_zone = subnet.availability_zone
        public            = subnet.public
        cidr              = local.subnet_cidrs[i]
      }
    ]
  ])


  private_subnets_by_az = { for s in local.subnets : s.availability_zone => s.cidr... if s.public == false }
  public_subnets_by_az  = { for s in local.subnets : s.availability_zone => s.cidr... if s.public == true }
}

resource "aws_subnet" "public_subnet" {
  count  = length(var.availability_zones)
  vpc_id = local.vpc_id
  cidr_block              = local.public_subnets_by_az[var.availability_zones[count.index]][0]
  availability_zone       = var.availability_zones[count.index]
  map_public_ip_on_launch = true
  tags = merge(
    {
      Name = "${var.cluster_name}-public-subnet-${count.index}"
    }
  )
}

resource "aws_subnet" "private_subnet" {
  count  = var.use_only_public_subnet ? 0 : length(var.availability_zones)
  vpc_id = local.vpc_id
  cidr_block              = local.private_subnets_by_az[var.availability_zones[count.index]][0]
  availability_zone       = var.availability_zones[count.index]
  map_public_ip_on_launch = false
  tags = merge(
    {
      Name = "${var.cluster_name}-private-subnet-${count.index}"
    }
  )
}

提问 #

调整AWS VPC可用区数量时频繁出现子网CIDR重叠错误，请问需要重点关注CIDR块计算中的哪些部分来避免这个问题？

1. 子网大小的计算逻辑 #

当前通过二进制格式化IP数量再推导CIDR大小的逻辑过于绕弯：

• floor(local.private_subnet_ips_num / length(var.availability_zones))可能导致IP分配不均，进而让CIDR大小计算出现偏差，最终导致cidrsubnets生成重叠块
• 建议直接用CIDR掩码（比如固定/27）计算，避免二进制转换带来的失误

2. cidrsubnets的参数顺序与子网映射 #

现有逻辑按子网大小排序后再映射CIDR，会打乱可用区的原始顺序：

• 比如原本按「AZ1私有、AZ1公有、AZ2私有、AZ2公有」的顺序分配CIDR，排序后变成所有大子网先分配，调整AZ数量时，原有子网的CIDR位置会被打乱，导致新旧CIDR重叠
• 应保持子网按「可用区+类型」的固定顺序生成，让cidrsubnets按稳定顺序分配块

3. 子网分组与索引的可靠性 #

private_subnets_by_az和public_subnets_by_az用...语法会将同AZ下的子网CIDR存入列表，但资源中直接取[0]：

• 如果计算逻辑出错生成多个同类型子网，会重复使用同一个CIDR，或取到错误CIDR导致重叠
• 需确保每个AZ下的同类型子网唯一，或明确处理多子网场景的索引逻辑

4. 调整AZ时的CIDR分配稳定性 #

当前完全依赖AZ数量动态分配CIDR，未预留固定范围给公有/私有子网：

• 比如2个AZ时，公有子网占前两个块，私有占后两个；改成3个AZ时，公有子网会重新分配前三个块，和原有两个公有子网的CIDR重叠
• 建议预先划分VPC CIDR为公有、私有两大块，再在各自块内按AZ数量分配子网，调整AZ时只会在各自块内调整，不会跨块重叠

内容的提问来源于stack exchange，提问作者spicklerickle