使用Terraform创建EKS集群时aws_eks_node_group创建失败求助

错误信息 #

aws_eks_node_group.cluster_eks_node_group: Still creating... [23m30s elapsed]
aws_eks_node_group.cluster_eks_node_group: Still creating... [23m40s elapsed]
╷
│ Error: waiting for EKS Node Group (dean-eks-test:workers) to create: unexpected state 'CREATE_FAILED', wanted target 'ACTIVE'. last error: 1 error occurred:
│       * i-05c6b71e578f30a7b, i-06fe96881274c1a8e, i-08089f6a26ba4af49, i-0e02d8a45086f02e7: NodeCreationFailure: Instances failed to join the kubernetes cluster
│ 
│   with aws_eks_node_group.cluster_eks_node_group,
│   on eks.tf line 108, in resource "aws_eks_node_group" "cluster_eks_node_group":
│  108: resource "aws_eks_node_group" "cluster_eks_node_group" {
│ 
╵
╷
│ Error: Failed to create deployment: Post "http://localhost/apis/apps/v1/namespaces/default/deployments": dial tcp 127.0.0.1:80: connect: connection refused
│ 
│   with kubernetes_deployment.nginx_deployment,
│   on eks.tf line 122, in resource "kubernetes_deployment" "nginx_deployment":
│  122: resource "kubernetes_deployment" "nginx_deployment":
│ 
╵

用户的Terraform配置文件 #

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}


resource "aws_iam_role" "cluster_iam_role" {
  name = "eks-cluster-role"
  path = "/"

  assume_role_policy = <<EOF
{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Effect": "Allow",
   "Principal": {
    "Service": "eks.amazonaws.com"
   },
   "Action": "sts:AssumeRole"
  }
 ]
}
EOF

}

resource "aws_iam_role_policy_attachment" "AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role    = aws_iam_role.cluster_iam_role.name
}
resource "aws_iam_role_policy_attachment" "AmazonEC2ContainerRegistryReadOnly-EKS" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role    = aws_iam_role.cluster_iam_role.name
}

resource "aws_iam_role" "workernodes_iam_role" {
  name = "eks-node-role"

  assume_role_policy = jsonencode({
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
    Version = "2012-10-17"
  })
}

resource "aws_iam_role_policy_attachment" "AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role    = aws_iam_role.workernodes_iam_role.name
}

resource "aws_iam_role_policy_attachment" "AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role    = aws_iam_role.workernodes_iam_role.name
}

resource "aws_iam_role_policy_attachment" "EC2InstanceProfileForImageBuilderECRContainerBuilds" {
  policy_arn = "arn:aws:iam::aws:policy/EC2InstanceProfileForImageBuilderECRContainerBuilds"
  role    = aws_iam_role.workernodes_iam_role.name
}

resource "aws_iam_role_policy_attachment" "AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role    = aws_iam_role.workernodes_iam_role.name
}

# VPC
resource "aws_vpc" "cluster_vpc" {
  cidr_block = "10.0.0.0/16"
}

# Subnets
resource "aws_subnet" "cluster_subnets" {
  count = 2

  vpc_id     = aws_vpc.cluster_vpc.id
  cidr_block = "10.0.${count.index}.0/24"

  tags = {
    "Name" = "Private ${count.index}"
  }
}

# Create EKS cluster
resource "aws_eks_cluster" "eks_cluster" {
  name = "dean-eks-test"

  role_arn = aws_iam_role.cluster_iam_role.arn

  vpc_config {
    subnet_ids = aws_subnet.cluster_subnets[*].id
  }
}

# Worker Nodes
resource "aws_eks_node_group" "cluster_eks_node_group" {
  cluster_name    = aws_eks_cluster.eks_cluster.name
  node_group_name = "workers"
  node_role_arn   = aws_iam_role.workernodes_iam_role.arn
  subnet_ids      = aws_subnet.cluster_subnets[*].id

  scaling_config {
    desired_size = 4
    max_size     = 4
    min_size     = 4
  }
}

# Kubernetes Deployment
resource "kubernetes_deployment" "nginx_deployment" {
  metadata {
    name = "nginx"
    labels = {
      app = "nginx"
    }
  }

  spec {
    replicas = 2

    selector {
      match_labels = {
        app = "nginx"
      }
    }

    template {
      metadata {
        labels = {
          app = "nginx"
        }
      }

      spec {
        container {
          image = "nginx:latest"
          name  = "nginx"

          port {
            container_port = 80
          }
        }
      }
    }
  }
}

# Kubernetes Load Balancer Service
resource "kubernetes_service" "nginx_service" {
  metadata {
    name = "nginx-lb"
  }
  spec {
    selector = {
      app = kubernetes_deployment.nginx_deployment.metadata[0].labels.app
    }
    port {
      port        = 80
      target_port = 80
    }

    type = "LoadBalancer"
  }
}

问题分析与修复方案 #

1. 节点无法加入集群的核心原因 #

• 私有子网无出站访问能力：当前使用的私有子网没有配置NAT网关/实例，节点无法访问EKS控制平面、AWS STS服务（获取角色凭证）或拉取节点镜像，导致无法完成集群注册。
• VPC DNS配置缺失：未启用VPC的私有DNS解析和主机名功能，节点无法解析EKS控制平面的域名。
• 冗余IAM权限：节点角色附加了与EKS无关的EC2InstanceProfileForImageBuilderECRContainerBuilds政策，虽不直接影响注册，但属于无效配置。

2. Kubernetes部署连接拒绝的原因 #

Terraform的Kubernetes Provider默认尝试连接本地localhost:80，但未配置正确的EKS集群访问凭证；同时资源执行顺序错误，K8s部署在EKS集群和节点组就绪前就开始执行。

具体修复步骤 #

（1）修复网络配置，确保节点有出站访问

添加互联网网关、公有子网、NAT网关，为私有子网配置路由：

# 互联网网关
resource "aws_internet_gateway" "cluster_igw" {
  vpc_id = aws_vpc.cluster_vpc.id
}

# 公有子网（用于部署NAT网关）
resource "aws_subnet" "public_subnets" {
  count = 2
  vpc_id                  = aws_vpc.cluster_vpc.id
  cidr_block              = "10.0.${count.index + 2}.0/24"
  map_public_ip_on_launch = true
  tags = {
    Name = "Public ${count.index}"
  }
}

# NAT网关EIP
resource "aws_eip" "nat_eip" {
  count = 2
  vpc   = true
}

# NAT网关
resource "aws_nat_gateway" "cluster_nat" {
  count = 2
  allocation_id = aws_eip.nat_eip[count.index].id
  subnet_id     = aws_subnet.public_subnets[count.index].id
}

# 私有子网路由表（指向NAT网关）
resource "aws_route_table" "private_route_table" {
  count = 2
  vpc_id = aws_vpc.cluster_vpc.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.cluster_nat[count.index].id
  }
}

# 关联私有子网到路由表
resource "aws_route_table_association" "private_subnet_assoc" {
  count = 2
  subnet_id      = aws_subnet.cluster_subnets[count.index].id
  route_table_id = aws_route_table.private_route_table[count.index].id
}

# 公有子网路由表（指向互联网网关）
resource "aws_route_table" "public_route_table" {
  vpc_id = aws_vpc.cluster_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.cluster_igw.id
  }
}

resource "aws_route_table_association" "public_subnet_assoc" {
  count = 2
  subnet_id      = aws_subnet.public_subnets[count.index].id
  route_table_id = aws_route_table.public_route_table.id
}

启用VPC的DNS功能：

resource "aws_vpc" "cluster_vpc" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true
}

（2）清理冗余IAM权限

删除节点角色上的无效政策附件：

# 移除以下不必要的配置
# resource "aws_iam_role_policy_attachment" "EC2InstanceProfileForImageBuilderECRContainerBuilds" {
#   policy_arn = "arn:aws:iam::aws:policy/EC2InstanceProfileForImageBuilderECRContainerBuilds"
#   role    = aws_iam_role.workernodes_iam_role.name
# }

（3）配置Kubernetes Provider依赖与访问凭证

添加Kubernetes Provider并关联EKS集群，确保资源执行顺序正确：

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
    kubernetes = {
      source = "hashicorp/kubernetes"
    }
  }
}

provider "kubernetes" {
  host                   = aws_eks_cluster.eks_cluster.endpoint
  cluster_ca_certificate = base64decode(aws_eks_cluster.eks_cluster.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.cluster_auth.token

  depends_on = [aws_eks_node_group.cluster_eks_node_group]
}

data "aws_eks_cluster_auth" "cluster_auth" {
  name = aws_eks_cluster.eks_cluster.name
}

为K8s资源添加依赖，确保节点组就绪后再执行：

resource "kubernetes_deployment" "nginx_deployment" {
  # 原有配置不变
  depends_on = [aws_eks_node_group.cluster_eks_node_group]
}

resource "kubernetes_service" "nginx_service" {
  # 原有配置不变
  depends_on = [kubernetes_deployment.nginx_deployment]
}

总结 #

核心问题是私有子网缺少出站访问能力，导致节点无法完成EKS集群注册；同时Kubernetes Provider未正确配置EKS集群连接信息，且资源执行顺序错误。修复网络配置、清理冗余权限、配置Provider依赖后即可解决问题。

内容的提问来源于stack exchange，提问作者Dean Schulze