CloudFormation创建EKS NodeGroup报错:实例无法加入Kubernetes集群
通过CloudFormation模板创建VPC、EKS集群及NodeGroup时,NodeGroup报错:
Resource handler returned message: "[Issue(Code=NodeCreationFailure, Message=Instances failed to join the kubernetes cluster, ResourceIds=[i-01e2bc499c53b0b40, i-0e2aca2058a6f0192])] (Service: null, Status Code: 0, Request ID: null)" (RequestToken: b376f260-d21e-cba9-f0e2-5f01c8f1d374, HandlerErrorCode: GeneralServiceException)
通过模板创建除NodeGroup外的其他资源后,手动在AWS控制台创建NodeGroup可正常加入集群。已预先创建可用IAM角色,相关模板如下:
Parameters: EnvironmentName: Description: An environment name that is prefixed to resource names Type: String Default: test-cloudformation VpcCIDR: Description: Please enter the IP range (CIDR notation) for this VPC Type: String Default: 10.192.0.0/16 PublicSubnet1CIDR: Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone Type: String Default: 10.192.10.0/24 PublicSubnet2CIDR: Description: Please enter the IP range (CIDR notation) for the public subnet in the second Availability Zone Type: String Default: 10.192.11.0/24 PrivateSubnet1CIDR: Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone Type: String Default: 10.192.20.0/24 PrivateSubnet2CIDR: Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone Type: String Default: 10.192.21.0/24 Resources: VPC: Type: AWS::EC2::VPC Properties: CidrBlock: !Ref VpcCIDR EnableDnsSupport: true EnableDnsHostnames: true Tags: - Key: Name Value: !Ref EnvironmentName InternetGateway: Type: AWS::EC2::InternetGateway Properties: Tags: - Key: Name Value: !Ref EnvironmentName InternetGatewayAttachment: Type: AWS::EC2::VPCGatewayAttachment Properties: InternetGatewayId: !Ref InternetGateway VpcId: !Ref VPC PublicSubnet1: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 0, !GetAZs '' ] CidrBlock: !Ref PublicSubnet1CIDR MapPublicIpOnLaunch: true Tags: - Key: Name Value: !Sub ${EnvironmentName} Public Subnet (AZ1) PublicSubnet2: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 1, !GetAZs '' ] CidrBlock: !Ref PublicSubnet2CIDR MapPublicIpOnLaunch: true Tags: - Key: Name Value: !Sub ${EnvironmentName} Public Subnet (AZ2) PrivateSubnet1: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 0, !GetAZs '' ] CidrBlock: !Ref PrivateSubnet1CIDR MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Subnet (AZ1) PrivateSubnet2: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC AvailabilityZone: !Select [ 1, !GetAZs '' ] CidrBlock: !Ref PrivateSubnet2CIDR MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Subnet (AZ2) NatGateway1EIP: Type: AWS::EC2::EIP DependsOn: InternetGatewayAttachment Properties: Domain: vpc NatGateway2EIP: Type: AWS::EC2::EIP DependsOn: InternetGatewayAttachment Properties: Domain: vpc NatGateway1: Type: AWS::EC2::NatGateway Properties: AllocationId: !GetAtt NatGateway1EIP.AllocationId SubnetId: !Ref PublicSubnet1 NatGateway2: Type: AWS::EC2::NatGateway Properties: AllocationId: !GetAtt NatGateway2EIP.AllocationId SubnetId: !Ref PublicSubnet2 PublicRouteTable: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub ${EnvironmentName} Public Routes DefaultPublicRoute: Type: AWS::EC2::Route DependsOn: InternetGatewayAttachment Properties: RouteTableId: !Ref PublicRouteTable DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref InternetGateway PublicSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PublicRouteTable SubnetId: !Ref PublicSubnet1 PublicSubnet2RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PublicRouteTable SubnetId: !Ref PublicSubnet2 PrivateRouteTable1: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Routes (AZ1) DefaultPrivateRoute1: Type: AWS::EC2::Route Properties: RouteTableId: !Ref PrivateRouteTable1 DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGateway1 PrivateSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PrivateRouteTable1 SubnetId: !Ref PrivateSubnet1 PrivateRouteTable2: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub ${EnvironmentName} Private Routes (AZ2) DefaultPrivateRoute2: Type: AWS::EC2::Route Properties: RouteTableId: !Ref PrivateRouteTable2 DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGateway2 PrivateSubnet2RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref PrivateRouteTable2 SubnetId: !Ref PrivateSubnet2 EKSClusterSG: Type: "AWS::EC2::SecurityGroup" Properties: GroupDescription: "Security Group for EKS Nodes" VpcId: !Ref VPC SecurityGroupIngress: - IpProtocol: tcp FromPort: 22 ToPort: 22 CidrIp: 0.0.0.0/0 # Adjust the CIDR to limit access as needed SecurityGroupEgress: - IpProtocol: "-1" FromPort: 0 ToPort: 0 CidrIp: 0.0.0.0/0 # Allow all outbound traffic Tags: - Key: "Name" Value: "EKSClusterSG" MyEKSCluster: Type: "AWS::EKS::Cluster" Properties: Name: !Sub "${EnvironmentName}-eks-cluster" RoleArn: "arn:aws:iam::0000000000:role/EKS" Version: "1.28" # Specify the desired Kubernetes version ResourcesVpcConfig: SecurityGroupIds: - !Ref EKSClusterSG SubnetIds: - !Ref PrivateSubnet1 - !Ref PrivateSubnet2 EndpointPublicAccess: true EndpointPrivateAccess: true CoreDNSAddon: Type: AWS::EKS::Addon Properties: AddonName: coredns ClusterName: !Ref MyEKSCluster AddonVersion: v1.10.1-eksbuild.6 # Specify the desired version compatible with your EKS cluster ResolveConflicts: OVERWRITE KubeProxyAddon: Type: AWS::EKS::Addon Properties: AddonName: kube-proxy ClusterName: !Ref MyEKSCluster AddonVersion: v1.28.1-eksbuild.1 # Specify the desired version compatible with your EKS cluster ResolveConflicts: OVERWRITE VpcCniAddon: Type: AWS::EKS::Addon Properties: AddonName: vpc-cni ClusterName: !Ref MyEKSCluster AddonVersion: v1.14.1-eksbuild.1 # Specify the desired version compatible with your EKS cluster ResolveConflicts: OVERWRITE EKSNodeGroup: Type: "AWS::EKS::Nodegroup" Properties: ClusterName: !Ref MyEKSCluster # This references your EKS Cluster defined above NodegroupName: !Sub "${EnvironmentName}-eks-nodegroup" NodeRole: "arn:aws:iam::0000000000:role/EKSworkers" # Replace with your Node IAM role ARN Subnets: - !Ref PrivateSubnet1 # Replace with your actual private subnet logical IDs - !Ref PrivateSubnet2 ScalingConfig: DesiredSize: 2 MinSize: 1 MaxSize: 3 InstanceTypes: - "t3.micro" DiskSize: 10 RemoteAccess: # Optional: Only necessary if you need to SSH into the nodes Ec2SshKey: "test-private" # Replace with your actual key pair name SourceSecurityGroups: - !Ref EKSClusterSG Labels: node-role.kubernetes.io/worker: "worker" AmiType: AL2_x86_64
调整资源依赖顺序:模板中NodeGroup仅依赖EKS集群,但集群创建完成后,CoreDNS、kube-proxy、VPC CNI等Addon可能未部署就绪,导致节点无法正常加入。手动创建时集群及Addon均已就绪,因此无问题。需给
EKSNodeGroup添加DependsOn属性,明确依赖三个Addon资源:EKSNodeGroup: Type: "AWS::EKS::Nodegroup" DependsOn: - CoreDNSAddon - KubeProxyAddon - VpcCniAddon Properties: # 原有属性保持不变补全安全组必要规则:当前
EKSClusterSG仅开放22端口入站,缺少EKS节点与集群控制平面通信的必要规则。需添加以下入站规则到EKSClusterSG:SecurityGroupIngress: # 原有22端口规则保留 - IpProtocol: tcp FromPort: 443 ToPort: 443 SourceSecurityGroupId: !Ref EKSClusterSG - IpProtocol: tcp FromPort: 10250 ToPort: 10250 SourceSecurityGroupId: !Ref EKSClusterSG这允许集群安全组内的流量(节点与控制平面)通过443(API服务器)和10250(节点kubelet)端口通信。
添加子网EKS专属标签:EKS节点所在私有子网需添加集群关联标签,控制台手动创建NodeGroup时会自动添加,但模板中未配置。给
PrivateSubnet1和PrivateSubnet2的Tags添加:Tags: # 原有Name标签保留 - Key: !Sub "kubernetes.io/cluster/${EnvironmentName}-eks-cluster" Value: "shared"验证节点IAM角色配置:确认
EKSworkers角色已附加以下托管策略:AmazonEKSWorkerNodePolicyAmazonEC2ContainerRegistryReadOnlyAmazonEKS_CNI_Policy
同时检查角色信任关系,确保包含ec2.amazonaws.com和eks.amazonaws.com为可信实体。
排查节点启动日志:登录报错的EC2实例,查看
/var/log/cloud-init-output.log和/var/log/messages日志,确认具体失败原因:- 是否无法解析集群API endpoint
- 是否IAM权限不足导致无法获取集群配置
- 是否网络连通性问题(如无法访问AWS服务或集群控制平面)
内容的提问来源于stack exchange,提问作者Dybbuk

