为Terraform Azurerm应用网关添加可信根证书时出错求助

问题背景 #

需要让K8s集群的Ingress指向集群内由非知名CA签发TLS证书的服务/Pod，因此在已有azurerm_application_gateway Terraform资源中添加trusted_root_certificate块，但执行terraform apply时失败，尽管可信根证书已成功添加到应用网关及后端设置中。

报错信息 #

Terraform执行错误 #

Error: waiting for update of Application Gateway: (Name "@@@" / Resource Group "@@@"): Code="ApplicationGatewayKeyVaultSecretException" Message="Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/@@@/resourceGroups/@@@/providers/Microsoft.Network/applicationGateways/@@@'. See details below:" Details=[{"code":"ApplicationGatewayTrustedRootCertificateInvalidData","message":"Data for certificate /subscriptions/@@@/resourceGroups/@@@/providers/Microsoft.Network/applicationGateways/@@@/trustedRootCertificates/@@@ is invalid."}]

Azure门户错误提示 #

Last configuration update operation on this Application Gateway failed. This will not impact the functioning of the Application Gateway and it will continue to serve your application traffic. If you intend to change the configuration of the Application Gateway, please try doing the configuration update again.

已尝试操作 #

• 在Key Vault中使用两种证书格式：含私钥和证书链的PEM、手动转换的PFX格式
• 配置trusted_root_certificate块关联Key Vault中的证书

相关配置 #

简化后的Terraform代码 #

locals {
  ca_name = "myca"
}

data "azurerm_key_vault_certificate" "ca" {
  name         = local.ca_name
  key_vault_id = data.azurerm_key_vault.myvault.id
}

resource "azurerm_application_gateway" "http_ingress" {
  sku {
    name     = "WAF_v2"
  }

  backend_http_settings {
    name                  = "https"
    cookie_based_affinity = "Disabled"
    port                  = 443
    protocol              = "Https"
    pick_host_name_from_backend_address = true
    trusted_root_certificate_names = ["${local.ca_name}"]
  }

  trusted_root_certificate {
    name                = local.ca_name
    key_vault_secret_id = data.azurerm_key_vault_certificate.ca.versionless_secret_id
  }
}

Terraform Provider配置 #

azurerm = {
  source  = "hashicorp/azurerm"
  version = "~> 3.78.0"
}

本地环境信息 #

terraform -v
Terraform v1.4.5
on darwin_arm64

解决思路 #

• 修正证书格式：可信根证书仅需CA的公钥部分，不能包含私钥。之前使用的PEM/PFX包含私钥，导致验证失败。提取CA证书的公钥PEM（仅保留-----BEGIN CERTIFICATE-----到-----END CERTIFICATE-----的内容），上传至Key Vault作为Secret（而非带私钥的Certificate资源）。
• 调整Terraform资源引用：改用data.azurerm_key_vault_secret引用Key Vault中的公钥秘密，避免引用包含私钥的证书资源。示例：

  data "azurerm_key_vault_secret" "ca_root" {
    name         = local.ca_name
    key_vault_id = data.azurerm_key_vault.myvault.id
  }
  
  resource "azurerm_application_gateway" "http_ingress" {
    # ...其他配置
    trusted_root_certificate {
      name                = local.ca_name
      key_vault_secret_id = data.azurerm_key_vault_secret.ca_root.versionless_id
    }
  }
  

• 验证Key Vault权限：确保应用网关的托管身份（或服务主体）拥有Key Vault的Secret Get和Certificate Get权限，避免因权限不足导致验证失败。
• 手动验证证书有效性：先在Azure门户手动添加CA根证书到应用网关，确认是否成功。若手动失败，说明证书本身存在格式或内容问题。
• 升级Terraform Provider版本：当前使用的3.78.0版本可能存在已知bug，尝试升级至3.80+的稳定版本后重新部署。
• 清理后重新部署：先移除Terraform中的trusted_root_certificate块，执行terraform apply修复应用网关的不一致状态，再重新添加正确配置的证书块进行部署。

内容的提问来源于stack exchange，提问作者toddmacintyre