如何通过Terraform终止处于Pending:Wait状态的Auto Scaling实例?
解决Terraform管理ASG时缩容/销毁卡住Pending:Wait实例的问题
我之前也踩过完全一样的坑——带启动生命周期钩子的ASG,缩容时只会碰健康实例,Pending:Wait的实例死活不动;销毁ASG时更头疼,没有健康实例的情况下ASG会不停创建新实例卡进Pending:Wait,Terraform设了期望数0也得等最后一个实例超时。下面分享两个针对性的解决方案:
一、销毁ASG时快速清理Pending:Wait实例
如果是销毁ASG(尤其是标记为tainted的场景),最直接的方式是在销毁前主动触发生命周期动作,让这些卡着的实例被终止。可以用Terraform的null_resource配合local-exec执行AWS CLI命令:
resource "null_resource" "cleanup_pending_asg_instances" { depends_on = [aws_autoscaling_group.my_asg] provisioner "local-exec" { command = <<EOT # 抓取当前ASG中处于Pending:Wait状态的实例ID INSTANCE_IDS=$(aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names ${aws_autoscaling_group.my_asg.name} --query 'AutoScalingGroups[0].Instances[?LifecycleState==`Pending:Wait`].InstanceId' --output text) # 如果存在这类实例,发送ABANDON信号让ASG直接终止它们 if [ -n "$INSTANCE_IDS" ]; then for ID in $INSTANCE_IDS; do aws autoscaling complete-lifecycle-action \ --auto-scaling-group-name ${aws_autoscaling_group.my_asg.name} \ --lifecycle-action-result ABANDON \ --instance-id $ID \ --lifecycle-hook-name ${aws_autoscaling_lifecycle_hook.my_launch_hook.name} done fi EOT when = destroy # 仅在销毁阶段执行该逻辑 } } # 确保ASG在清理完成后再开始销毁 resource "aws_autoscaling_group" "my_asg" { # ... 你的ASG基础配置 ... depends_on = [null_resource.cleanup_pending_asg_instances] }
这个脚本会在销毁ASG前主动找到所有卡壳的实例,告诉ASG放弃这些实例的启动流程,不用等钩子超时就能直接终止,大幅缩短销毁等待时间。
二、日常缩容时允许终止Pending:Wait实例
如果希望日常缩容(降低期望实例数)时,ASG能优先清理这些未正常启动的实例,可以通过「生命周期钩子+Lambda函数」实现智能判断:
1. 编写Lambda函数处理缩容场景
写一个简单的Lambda函数,当收到生命周期钩子的通知时,检查ASG的期望实例数:如果当前实例数多于期望数(说明在缩容),就自动放弃Pending:Wait的实例:
import boto3 def lambda_handler(event, context): asg_client = boto3.client('autoscaling') asg_name = event['AutoScalingGroupName'] instance_id = event['EC2InstanceId'] hook_name = event['LifecycleHookName'] # 获取ASG当前实例数和期望容量 asg_details = asg_client.describe_auto_scaling_groups(AutoScalingGroupNames=[asg_name])['AutoScalingGroups'][0] current_instance_count = len(asg_details['Instances']) desired_capacity = asg_details['DesiredCapacity'] # 缩容场景:主动放弃Pending:Wait实例 if current_instance_count > desired_capacity: asg_client.complete_lifecycle_action( AutoScalingGroupName=asg_name, InstanceId=instance_id, LifecycleActionResult='ABANDON', LifecycleHookName=hook_name ) else: # 正常扩容/启动场景:保持等待(可根据业务逻辑调整) pass
2. 在Terraform中配置钩子关联Lambda
# 创建Lambda所需的IAM角色,允许操作ASG resource "aws_iam_role" "asg_hook_lambda_role" { name = "asg-hook-lambda-role" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = "sts:AssumeRole" Effect = "Allow" Principal = { Service = "lambda.amazonaws.com" } } ] }) } resource "aws_iam_role_policy_attachment" "lambda_basic_execution" { role = aws_iam_role.asg_hook_lambda_role.name policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" } resource "aws_iam_role_policy" "asg_hook_access" { name = "asg-hook-access" role = aws_iam_role.asg_hook_lambda_role.id policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "autoscaling:DescribeAutoScalingGroups", "autoscaling:CompleteLifecycleAction" ] Effect = "Allow" Resource = "*" } ] }) } # 部署Lambda函数 resource "aws_lambda_function" "asg_scaling_handler" { filename = "lambda_function.zip" # 你的Lambda代码压缩包路径 function_name = "asg-scaling-handler" role = aws_iam_role.asg_hook_lambda_role.arn handler = "lambda_function.lambda_handler" runtime = "python3.11" } # 配置启动生命周期钩子 resource "aws_autoscaling_lifecycle_hook" "my_launch_hook" { name = "launch-wait-hook" auto_scaling_group_name = aws_autoscaling_group.my_asg.name lifecycle_transition = "autoscaling:EC2_INSTANCE_LAUNCHING" heartbeat_timeout = 3600 # 日常正常超时时间 default_result = "CONTINUE" # 超时后自动继续,避免无限卡住 notification_target_arn = aws_lambda_function.asg_scaling_handler.arn role_arn = aws_iam_role.asg_hook_lambda_role.arn }
这样配置后,当ASG处于缩容状态时,Lambda会自动识别并终止那些Pending:Wait的实例,不会让它们占用资源拖慢缩容流程。
额外小贴士
- 执行
local-exec的环境要配置好AWS凭证(比如环境变量、~/.aws/credentials,或者在EC2/ECS上运行时使用IAM角色)。 - 建议把生命周期钩子的
default_result设为CONTINUE,即使钩子超时,ASG也会继续处理实例,避免长期卡住。
内容的提问来源于stack exchange,提问作者wingedsubmariner
相关产品推荐
相关产品推荐

