如何使用Terraform强制删除Azure中被引用的Network Security Group？

问题描述 #

执行terraform destroy时遇到以下错误：

│ Error: deleting Network Security Group "multisqlite-bacalhau-eastus-ssh-nsg" (Resource Group "multisqlite-bacalhau-eastus-rg"): network.SecurityGroupsClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="NetworkSecurityGroupOldReferencesNotCleanedUp" Message="Network security group multisqlite-bacalhau-eastus-ssh-nsg cannot be deleted because old references for the following Nics: (\n/subscriptions/72ac7288-fb92-4ad6-83bc-5cfd361f47ef/resourceGroups/multisqlite-bacalhau-eastus-rg/providers/Microsoft.Network/networkSecurityGroups/multisqlite-bacalhau-eastus-ssh-nsg:/subscriptions/72ac7288-fb92-4ad6-83bc-5cfd361f47ef/resourceGroups/multisqlite-bacalhau-eastus-rg/providers/Microsoft.Network/networkInterfaces/multisqlite-bacalhau-eastus-nic) and Subnet: (\n/subscriptions/72ac7288-fb92-4ad6-83bc-5cfd361f47ef/resourceGroups/multisqlite-bacalhau-eastus-rg/providers/Microsoft.Network/networkSecurityGroups/multisqlite-bacalhau-eastus-ssh-nsg:) have not been released yet." Details=[]

错误提示NSG因网卡和子网的旧引用未释放无法删除，但通过Azure门户可正常删除，删除后Terraform能完成销毁流程。

原因分析 #

1. 销毁顺序不匹配：默认依赖关系下，NSG可能在关联的VM、NIC资源完全销毁前就被触发删除，而Azure后台的关联引用缓存未及时清理。
2. Provider版本缺陷：旧版本AzureRM Provider存在资源销毁时的依赖处理bug，导致关联关系未正确解除。
3. Data资源延迟：配置中的data "azurerm_public_ip"依赖已销毁的VM，可能导致Terraform在销毁阶段仍持有旧资源引用。

解决方案 #

1. 调整资源销毁依赖顺序 #

确保NSG在关联VM、NIC完全销毁后再执行删除。修改azurerm_network_security_group资源，添加depends_on：

resource "azurerm_network_security_group" "nsg" {
  for_each            = var.locations
  name                = "${var.app_tag}-${each.key}-ssh-nsg"
  location            = azurerm_resource_group.rg[each.key].location
  resource_group_name = azurerm_resource_group.rg[each.key].name

  security_rule {
    name                       = "allow_ssh_sg"
    priority                   = 100
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_range     = "22"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }

  # 确保NSG在VM销毁后再删除
  depends_on = [azurerm_linux_virtual_machine.instance[each.key]]
}

2. 升级AzureRM Provider版本 #

将Provider更新至最新3.x稳定版，修复已知销毁依赖bug：

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>3.80" # 替换为最新稳定版
    }
  }
}

执行terraform init -upgrade完成升级。

3. 销毁前手动解除NSG关联（临时方案） #

通过null_resource调用Azure CLI提前解除NIC与NSG的关联：

resource "null_resource" "detach_nsg_before_destroy" {
  for_each = var.locations
  triggers = {
    nsg_id = azurerm_network_security_group.nsg[each.key].id
    nic_id = azurerm_network_interface.nic[each.key].id
  }

  provisioner "local-exec" {
    when        = destroy
    command     = "az network nic update --ids ${each.value.nic_id} --remove networkSecurityGroup"
    environment = {
      AZURE_SUBSCRIPTION_ID = var.subscriptionId
    }
  }

  depends_on = [azurerm_linux_virtual_machine.instance[each.key]]
}

# 让NSG依赖该null_resource
resource "azurerm_network_security_group" "nsg" {
  # ... 现有配置 ...
  depends_on = [null_resource.detach_nsg_before_destroy[each.key]]
}

注意：执行环境需安装Azure CLI并配置好权限。

4. 移除不必要的Data资源依赖 #

将data "azurerm_public_ip"替换为直接引用资源输出，避免依赖已销毁的VM：

# 删除原data资源
# data "azurerm_public_ip" "public_ip" { ... }

# 修改locals和null_resource中的引用
locals {
  bootstrap_ip = azurerm_public_ip.public_ip_allocator[var.bootstrap_region].ip_address
}

resource "null_resource" "copy-to-node-if-worker" {
  for_each = local.non_bootstrap_regions

  connection {
    host        = azurerm_public_ip.public_ip_allocator[each.value].ip_address
    # ... 其他配置 ...
  }

  # ... 其他配置 ...
}

5. 强制删除NSG（应急方案） #

若上述方法无效，先通过Azure CLI手动删除NSG，再继续Terraform销毁：

az network nsg delete --name multisqlite-bacalhau-eastus-ssh-nsg --resource-group multisqlite-bacalhau-eastus-rg

执行后运行terraform destroy，Terraform会检测NSG已删除并继续处理其他资源。

内容的提问来源于stack exchange，提问作者aronchick