配置带KMS加密卷的Spot Fleet请求时实例启动失败求助
ECS集群Spot实例启动失败:
Client.InternalError: Client error on launch 问题排查与修复 尝试为ECS集群启动Spot实例时持续失败,实例启动阶段报错Client.InternalError: Client error on launch。根据官方文档提示,该问题由EBS卷的IAM权限问题导致。已尝试为Fleet角色和实例角色赋予尽可能多的权限(怀疑实例角色是关键,但不确定),但问题仍未解决。以下是最小复现代码(MRE):
最小复现代码(HCL)
IAM 资源
resource "aws_iam_role" "fleet" { name = "ecs-fleet" assume_role_policy = jsonencode( { "Version" : "2008-10-17", "Statement" : [ { "Sid" : "", "Effect" : "Allow", "Principal" : { "Service" : [ "spotfleet.amazonaws.com", "ec2.amazonaws.com" ] }, "Action" : "sts:AssumeRole" } ] }) } resource "aws_iam_role" "ecs" { name = "ecs.role" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = "sts:AssumeRole" Effect = "Allow" Sid = "" Principal = { Service = "ec2.amazonaws.com" } }, ] }) } resource "aws_iam_instance_profile" "ecs" { name = "ecs.profile" role = aws_iam_role.ecs.name } resource "aws_iam_policy_attachment" "fleet" { name = "ecs-fleet" roles = [aws_iam_role.fleet.name] policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2SpotFleetTaggingRole" }
KMS 密钥资源
resource "aws_kms_key" "default" { policy = jsonencode( { "Version" : "2012-10-17", "Id" : "key-default-1", "Statement" : [ { "Sid" : "Allow service-linked role use of the customer managed key", "Effect" : "Allow", "Principal" : { "AWS" : [ aws_iam_role.ecs.arn, aws_iam_role.fleet.arn, ] }, "Action" : [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ], "Resource" : "*" }, { "Sid" : "Allow attachment of persistent resources", "Effect" : "Allow", "Principal" : { "AWS" : [ aws_iam_role.ecs.arn, aws_iam_role.fleet.arn ] }, "Action": [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ], "Resource" : "*", "Condition" : { "Bool" : { "kms:GrantIsForAWSResource" : "true" } }, } ] } ) }
Spot实例相关资源
resource "aws_launch_template" "ecs_c6i" { name = "ECS-c6i.xlarge" image_id = "ami-00f05e43221bf6048" # al2022-ami-ecs-hvm-2022.0.20220630-x86_64-ebs ebs_optimized = true instance_type = "c6i.xlarge" iam_instance_profile { name = aws_iam_instance_profile.ecs.name } block_device_mappings { device_name = "/dev/xvda" ebs { kms_key_id = aws_kms_key.default.arn volume_size = 30 delete_on_termination = true encrypted = true volume_type = "gp3" } } } resource "aws_spot_fleet_request" "main" { iam_fleet_role = aws_iam_role.fleet.arn allocation_strategy = "lowestPrice" target_capacity = 1 terminate_instances_with_expiration = false launch_template_config { launch_template_specification { id = aws_launch_template.ecs_c6i.id version = "$Latest" } } }
问题根源与修复方案
核心问题
报错的直接原因是Spot Fleet角色缺少操作KMS密钥的权限,同时KMS密钥策略未允许EC2服务关联角色访问密钥——当Spot Fleet创建加密EBS卷时,需要同时具备这两类权限才能成功挂载卷并启动实例。
修复步骤
为Spot Fleet角色添加KMS权限
当前Fleet角色仅附加了AmazonEC2SpotFleetTaggingRole策略,该策略不包含KMS操作权限。需创建自定义IAM策略并附加到Fleet角色:# 获取当前AWS账号ID data "aws_caller_identity" "current" {} # 创建KMS访问策略 resource "aws_iam_policy" "fleet_kms" { name = "ecs-fleet-kms-access" description = "允许Spot Fleet使用指定KMS密钥进行EBS加密" policy = jsonencode({ Version = "2012-10-17" Statement = [ { Effect = "Allow" Action = [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ], Resource = aws_kms_key.default.arn } ] }) } # 将策略附加到Fleet角色 resource "aws_iam_policy_attachment" "fleet_kms" { name = "ecs-fleet-kms-attachment" roles = [aws_iam_role.fleet.name] policy_arn = aws_iam_policy.fleet_kms.arn }更新KMS密钥策略,添加EC2服务关联角色
EC2服务在创建加密EBS卷时,会使用自身的服务关联角色AWSServiceRoleForEC2操作KMS密钥,需将该角色加入KMS策略的允许列表:
修改aws_kms_key.default的policy部分,在两个Statement的Principal.AWS数组中添加以下ARN:"arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/aws-service-role/ec2.amazonaws.com/AWSServiceRoleForEC2"修改后的KMS策略Principal部分示例:
"Principal" : { "AWS" : [ aws_iam_role.ecs.arn, aws_iam_role.fleet.arn, "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/aws-service-role/ec2.amazonaws.com/AWSServiceRoleForEC2" ] }验证实例角色权限(可选)
若后续实例需要直接访问KMS加密资源,可给ECS实例角色添加对应的KMS权限,但这并非本次启动报错的必要条件。
内容的提问来源于stack exchange,提问作者RobinFrcd
相关产品推荐
相关产品推荐

