如何让Terraform等待user_data执行完成后制作镜像用于自动扩容？

一、先解决SSM命令执行的错误 #

你遇到的InvalidInstanceId错误，核心原因有两个：实例刚创建未进入可用状态、实例缺少SSM管理的权限或Agent未就绪。

1. 给实例附加SSM管理的IAM角色 #

实例需要具备SSM访问权限才能接收命令，创建以下IAM资源：

# SSM实例角色
resource "aws_iam_role" "ssm_instance_role" {
  name = "ssm-instance-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      }
    ]
  })
}

# 附加SSM核心权限策略
resource "aws_iam_role_policy_attachment" "ssm_core" {
  role       = aws_iam_role.ssm_instance_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

# 实例配置文件
resource "aws_iam_instance_profile" "ssm_instance_profile" {
  name = "ssm-instance-profile"
  role = aws_iam_role.ssm_instance_role.name
}

然后在实例资源中关联该配置文件：

resource "aws_instance" "example" {
  ami           = var.instance_ami
  instance_type = "t2.micro"
  key_name      = var.ssh_key
  # 新增：关联SSM实例配置文件
  iam_instance_profile = aws_iam_instance_profile.ssm_instance_profile.name

  # ... 其他原有配置
}

2. 完善等待逻辑，确保实例就绪 #

修改local-exec的命令，先等待实例进入running状态，再等待SSM Agent在线，最后执行Cloud-Init等待命令：

provisioner "local-exec" {
  interpreter = ["/bin/bash", "-c"]

  command = <<-EOF
    set -Ee -o pipefail
    export AWS_DEFAULT_REGION=${var.aws_region}

    # 等待实例进入running状态
    aws ec2 wait instance-running --instance-ids ${self.id}

    # 轮询等待SSM Agent在线
    until aws ssm describe-instance-information --filters "Key=InstanceIds,Values=${self.id}" --query "InstanceInformationList[*].PingStatus" --output text | grep -q "Online"; do
      echo "Waiting for SSM Agent on ${self.id} to come online..."
      sleep 10
    done

    # 发送SSM命令等待cloud-init完成
    command_id=$(aws ssm send-command --document-name ${aws_ssm_document.cloud_init_wait.name} --instance-ids ${self.id} --output text --query "Command.CommandId")
    if ! aws ssm wait command-executed --command-id $command_id --instance-id ${self.id}; then
      echo "Failed to wait for cloud-init on instance ${self.id}!";
      echo "stdout:";
      aws ssm get-command-invocation --command-id $command_id --instance-id ${self.id} --query StandardOutputContent --output text;
      echo "stderr:";
      aws ssm get-command-invocation --command-id $command_id --instance-id ${self.id} --query StandardErrorContent --output text;
      exit 1;
    fi;
    echo "Cloud-init completed successfully on instance ${self.id}!"
  EOF
}

二、制作Golden AMI并用于自动扩容 #

当Cloud-Init执行完成后，基于实例创建AMI，再将该AMI用于启动模板和自动扩容组：

1. 创建Golden AMI #

resource "aws_ami" "golden_image" {
  name                = "golden-media-processor-${timestamp()}"
  description         = "Pre-configured media processor image"
  source_instance_id  = aws_instance.example.id
  virtualization_type = "hvm"

  # 依赖实例的provisioner完成，确保Cloud-Init执行完毕
  depends_on = [aws_instance.example.provisioner.local-exec]

  tags = {
    Name = "Golden Media Processor AMI"
  }
}

2. 基于AMI创建启动模板与自动扩容组 #

# 启动模板
resource "aws_launch_template" "asg_template" {
  name_prefix   = "media-processor-launch-template-"
  image_id      = aws_ami.golden_image.id
  instance_type = "t2.micro"
  key_name      = var.ssh_key
  # 无需再配置user_data，AMI已包含初始化后的环境
}

# 自动扩容组
resource "aws_autoscaling_group" "media_processor_asg" {
  name_prefix          = "media-processor-asg-"
  min_size             = 1
  max_size             = 5
  desired_capacity     = 2
  vpc_zone_identifier  = var.subnet_ids

  launch_template {
    id      = aws_launch_template.asg_template.id
    version = "$Latest"
  }

  # 关联你的CloudWatch Alarm扩容缩容规则
  # ... 原有CloudWatch Alarm配置
}

三、替代方案：通过SSH直接等待Cloud-Init #

如果不想依赖SSM，可使用remote-exec通过SSH连接实例等待Cloud-Init完成：

resource "aws_instance" "example" {
  ami           = var.instance_ami
  instance_type = "t2.micro"
  key_name      = var.ssh_key
  vpc_security_group_ids = [var.security_group_id] # 确保开放SSH端口

  provisioner "remote-exec" {
    inline = [
      "cloud-init status --wait",
      "echo 'Cloud-init completed successfully'"
    ]

    connection {
      type        = "ssh"
      user        = "ec2-user" # 根据AMI调整，Ubuntu为ubuntu
      private_key = file(var.ssh_private_key_path)
      host        = self.public_ip
    }
  }
}

关键注意事项 #

• 确保使用的AWS官方AMI预装了SSM Agent（Amazon Linux 2、Ubuntu 20.04+等均默认包含）
• 安全组需允许实例与SSM服务通信（出站HTTPS 443）
• 若使用SSH方案，需确保实例有公网IP或通过VPC端点访问，安全组开放22端口

内容的提问来源于stack exchange，提问作者ashraf minhaj