通过Terraform启动模板创建EKS节点失败,aws-node Pod连接超时
问题现象
正常创建EKS节点组时一切正常,但通过Terraform自定义启动模板创建节点后,集群内出现连接问题,aws-node Pod报错:
{"level":"info","caller":"/usr/local/go/src/runtime/proc.go:225","msg":"timeout: failed to connect service ":50051" within 5s"}
已排除IAM角色问题(该角色曾成功创建其他节点),以下是相关Terraform配置:
EKS节点组配置
resource "aws_eks_node_group" "eth-staking-nodes" { cluster_name = aws_eks_cluster.staking.name node_group_name = "ethstaking-nodes-testnet" node_role_arn = aws_iam_role.nodes.arn subnet_ids = [ data.aws_subnet.private-1.id, data.aws_subnet.private-2.id ] scaling_config { desired_size = 1 max_size = 5 min_size = 0 } update_config { max_unavailable = 1 } labels = { role = "general" } launch_template { version = aws_launch_template.staking.latest_version id = aws_launch_template.staking.id } depends_on = [ aws_iam_role_policy_attachment.nodes-AmazonEKSWorkerNodePolicy, aws_iam_role_policy_attachment.nodes-AmazonEKS_CNI_Policy, aws_iam_role_policy_attachment.nodes-AmazonEC2ContainerRegistryReadOnly, ] }
启动模板配置
resource "aws_launch_template" "staking" { name = "${var.stage}-staking-node-launch-template" instance_type = "m5.2xlarge" image_id = "ami-08712c7468e314435" key_name = "nivpem" block_device_mappings { device_name = "/dev/xvda" ebs { volume_size = 450 volume_type = "gp2" } } lifecycle { create_before_destroy = false } vpc_security_group_ids = [aws_security_group.eks-ec2-sg.id] user_data = base64encode(templatefile("${path.module}/staking_userdata.sh", { password = "********" })) tags = { "eks:cluster-name" = aws_eks_cluster.staking.name "eks:nodegroup-name" = "ethstaking-nodes-testnet" } tag_specifications { resource_type = "instance" tags = { Name = "${var.stage}-staking-node" "eks:cluster-name" = aws_eks_cluster.staking.name "eks:nodegroup-name" = "ethstaking-nodes-testnet" } } }
安全组配置
resource "aws_security_group" "eks-ec2-sg" { name = "eks-ec2-sg-staking-testnet" vpc_id = data.aws_vpc.vpc.id ingress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] } egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] ipv6_cidr_blocks = ["::/0"] } tags = { Name = "allow_tls" } }
问题分析与修复方案
50051端口是EKS节点上CNI组件(aws-node)与本地服务的通信端口,超时通常由节点初始化不完整、标签缺失或AMI兼容性问题导致,以下是针对性修复:
1. 补全EKS节点初始化用户数据
自定义user_data覆盖了EKS默认的节点初始化脚本,导致CNI组件未正确配置。需在自定义脚本末尾添加官方初始化逻辑:
修改staking_userdata.sh:
#!/bin/bash # 你的自定义配置(比如设置密码) echo "root:${password}" | chpasswd # 加入EKS节点初始化脚本,自动配置CNI和集群连接 /etc/eks/bootstrap.sh ${cluster_name} --kubelet-extra-args '--node-labels=role=general'
同时在Terraform启动模板中传递集群名称变量:
user_data = base64encode(templatefile("${path.module}/staking_userdata.sh", { password = "********" cluster_name = aws_eks_cluster.staking.name }))
2. 添加集群归属标签
EKS需要通过kubernetes.io/cluster/${cluster_name}=owned标签识别节点归属,在启动模板的实例标签中补充该字段:
tag_specifications { resource_type = "instance" tags = { Name = "${var.stage}-staking-node" "eks:cluster-name" = aws_eks_cluster.staking.name "eks:nodegroup-name" = "ethstaking-nodes-testnet" "kubernetes.io/cluster/${aws_eks_cluster.staking.name}" = "owned" } }
3. 验证AMI兼容性
确认使用的ami-08712c7468e314435是对应区域的EKS优化AMI,非官方AMI可能缺少CNI依赖。可通过AWS CLI获取对应集群版本的官方AMI:
aws ssm get-parameter --name /aws/service/eks/optimized-ami/1.27/amazon-linux-2/recommended/image_id --region 你的集群区域
替换为实际集群版本和区域后,更新启动模板的image_id。
4. 确认私有子网网络连通性
检查节点所在私有子网是否配置NAT网关,确保节点能访问AWS EKS API、ECR等服务,这是CNI组件初始化的必要条件。
内容的提问来源于stack exchange,提问作者Niv Shitrit

