使用Terraform部署AKS集群后Nginx Ingress负载均衡IP pending问题求助

我尝试用Terraform创建AKS集群，部署基础负载均衡器，让Nginx-ingress-controller使用该负载均衡器，但查看nginx-ingress命名空间下的服务时，负载均衡器的外部IP一直处于<pending>状态：

NAME                                                          TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)                      AGE
nginx-ingress-controller-ingress-nginx-controller             LoadBalancer   10.0.60.72    <pending>     80:32183/TCP,443:31864/TCP   27s

执行kubectl get events的输出：

41s         Warning   SyncLoadBalancerFailed   service/nginx-ingress-controller-ingress-nginx-controller                 Error syncing load balancer: failed to ensure load balancer: instance not found

我原以为是AKS集群和负载均衡器不在同一资源组导致的，但实际并非如此，以下是我的Terraform代码：

# Set default name prefix
variable "name_prefix" {
  default = "Dev"
}

# Set default location
variable "location" {
  default = "australiacentral"
}

# Create Resource Group
resource "azurerm_resource_group" "aks_rg" {
  name     = "${var.name_prefix}-rg"
  location = var.location
}

# Managed Cluster resource group
#export MC_RESOURCE_GROUP=MC_${RESOURCE_GROUP}_${AKS_CLUSTER}_${LOCATION}

resource "azurerm_kubernetes_cluster" "dk_dev_cluster" {
  name                = var.name_prefix
  location            = azurerm_resource_group.aks_rg.location
  resource_group_name = azurerm_resource_group.aks_rg.name
  dns_prefix = "Dev"
  kubernetes_version = "1.25.5"
  # Create a basic load-balancer to lower costs
  network_profile {
    network_plugin = "azure"
    load_balancer_sku = "basic"
  }

  identity {
    type = "SystemAssigned"
  }

  default_node_pool {
    name    = lower("${var.name_prefix}nodep")
    vm_size = "Standard_B2s"
    node_count = 1
    enable_auto_scaling = false
  }

  tags = {
    Environment = "Dev"
  }
}

resource "azurerm_public_ip" "lb_pub_ip" {
  name                = "${var.name_prefix}-lb-ip"
  location            = var.location
  allocation_method   = "Static"
  sku = "Basic"
  resource_group_name = azurerm_kubernetes_cluster.dk_dev_cluster.node_resource_group
}

resource "azurerm_lb" "aks_lb" {
  name                = "${var.name_prefix}-lb"
  sku                 = "Basic"
  location            = var.location
  resource_group_name = azurerm_kubernetes_cluster.dk_dev_cluster.node_resource_group
  frontend_ip_configuration {
    name                 = "PublicIPAddress"
    public_ip_address_id = azurerm_public_ip.lb_pub_ip.id
  }
}

output "client_certificate" {
  value     = azurerm_kubernetes_cluster.dk_dev_cluster.kube_config.0.client_certificate
  sensitive = true
}

output "kube_config" {
  value = azurerm_kubernetes_cluster.dk_dev_cluster.kube_config_raw

  sensitive = true
}

# Create a kubeconfig file that can be added to/replace ~/.kube/config
resource "local_file" "kubeconfig" {
  depends_on   = [azurerm_kubernetes_cluster.dk_dev_cluster]
  filename     = "kubeconfig"
  content      = azurerm_kubernetes_cluster.dk_dev_cluster.kube_config_raw
}

provider "kubernetes" {
  config_path = local_file.kubeconfig.filename
  config_context = "Dev"
}

resource "kubernetes_namespace" "nginx_namespace" {
  metadata {
    name = "nginx-ingress"
  }
  depends_on = [local_file.kubeconfig]
}

provider "helm" {
}

resource "helm_release" "nginx_ingress" {
  name       = "nginx-ingress-controller"

  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"
  version    = "4.5.2"
  namespace = kubernetes_namespace.nginx_namespace.id

  set {
    name  = "controller.service.loadBalancerIP"
    value = azurerm_public_ip.lb_pub_ip.ip_address
  }
}

解决方案 #

问题核心并非资源组不匹配，而是手动创建的负载均衡器未被AKS云控制器管理器正确识别，且Nginx Ingress的LoadBalancer服务配置有误。以下提供两种可行解决方式：

方式一：复用AKS默认基础负载均衡器（推荐） #

AKS集群创建时会自动生成基础负载均衡器，无需手动创建额外LB，只需将Nginx Ingress绑定到默认LB的公网IP即可：

1. 删除Terraform中手动定义的azurerm_public_ip.lb_pub_ip和azurerm_lb.aks_lb资源。
2. 修正Helm release的配置：将namespace字段改为命名空间名称（而非ID），并绑定AKS默认公网IP：

resource "helm_release" "nginx_ingress" {
  name       = "nginx-ingress-controller"
  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"
  version    = "4.5.2"
  namespace  = kubernetes_namespace.nginx_namespace.metadata[0].name

  set {
    name  = "controller.service.loadBalancerIP"
    value = azurerm_kubernetes_cluster.dk_dev_cluster.public_ip_address
  }
}

方式二：手动创建LB并配置权限（自定义场景） #

若坚持使用手动创建的LB，需确保AKS系统标识有管理LB的权限，并配置Helm关联现有LB：

1. 给AKS系统标识授予托管资源组的网络参与者权限：

resource "azurerm_role_assignment" "aks_network_contributor" {
  scope                = azurerm_kubernetes_cluster.dk_dev_cluster.node_resource_group_id
  role_definition_name = "Network Contributor"
  principal_id         = azurerm_kubernetes_cluster.dk_dev_cluster.identity[0].principal_id
}

2. 修正Helm release配置，添加Azure LB关联注解，并依赖权限分配完成：

resource "helm_release" "nginx_ingress" {
  name       = "nginx-ingress-controller"
  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"
  version    = "4.5.2"
  namespace  = kubernetes_namespace.nginx_namespace.metadata[0].name
  depends_on = [azurerm_role_assignment.aks_network_contributor]

  set {
    name  = "controller.service.loadBalancerIP"
    value = azurerm_public_ip.lb_pub_ip.ip_address
  }

  # 注解指定LB所在资源组，让AKS云控制器识别现有LB
  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-resource-group"
    value = azurerm_kubernetes_cluster.dk_dev_cluster.node_resource_group
  }
}

额外注意事项

• 基础SKU负载均衡器仅支持关联一个Kubernetes LoadBalancer服务，若AKS默认服务已占用该LB，需先删除默认服务或改用标准SKU LB。
• 确保Helm版本与Ingress-Nginx chart版本兼容，避免因版本不匹配导致的配置问题。

内容的提问来源于stack exchange，提问作者Anon957