无法通过SSM参数作为凭据启动ECS任务的问题求助

报错信息 #

ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secrets from ssm: service call has been retried 5 time(s): RequestCanceled: request context canceled caused by: context deadline exceeded. Please check your task network configuration.

现有配置 #

VPC CloudFormation配置 #

MyVPC:
  Type: AWS::EC2::VPC
  Properties:
    CidrBlock: "10.0.0.0/16"
    EnableDnsHostnames: true
    EnableDnsSupport: true
MyDefaultRouteTable:
  DeletionPolicy: Retain
  Type: AWS::EC2::RouteTable
  Properties:
    VpcId: !Ref MyVPC
MyRouteTableA:
  Type: AWS::EC2::RouteTable
  Properties:
    VpcId: !Ref MyVPC
MyRouteTableB:
  Type: AWS::EC2::RouteTable
  Properties:
    VpcId: !Ref MyVPC
MyPublicSubnetA:
  Type: AWS::EC2::Subnet
  Properties:
    AvailabilityZone: "us-east-2a"
    CidrBlock: "10.0.0.0/24"
    MapPublicIpOnLaunch: true
    VpcId: !Ref MyVPC
MyPublicSubnetB:
  Type: AWS::EC2::Subnet
  Properties:
    AvailabilityZone: "us-east-2b"
    CidrBlock: "10.0.2.0/24"
    MapPublicIpOnLaunch: true
    VpcId: !Ref MyVPC
MyPrivateSubnetA:
  Type: AWS::EC2::Subnet
  Properties:
    AvailabilityZone: "us-east-2a"
    CidrBlock: "10.0.1.0/24"
    MapPublicIpOnLaunch: false
    VpcId: !Ref MyVPC
MyPrivateSubnetB:
  Type: AWS::EC2::Subnet
  Properties:
    AvailabilityZone: "us-east-2b"
    CidrBlock: "10.0.3.0/24"
    MapPublicIpOnLaunch: false
    VpcId: !Ref MyVPC
MyInternetGateway:
  Type: AWS::EC2::InternetGateway
  Properties:
    Tags:
      - Key: Env
        Value: production
MyInternetGatewayAttachment:
  Type: AWS::EC2::VPCGatewayAttachment
  Properties:
    InternetGatewayId: !Ref MyInternetGateway
    VpcId: !Ref MyVPC
MyNatGatewayEIPA:
  Type: AWS::EC2::EIP
  Properties:
    Domain: vpc
MyNatGatewayA:
  Type: AWS::EC2::NatGateway
  Properties:
    AllocationId: !GetAtt MyNatGatewayEIPA.AllocationId
    ConnectivityType: public
    SubnetId: !Ref MyPrivateSubnetA
MyNatGatewayRouteA:
  Type: AWS::EC2::Route
  Properties:
    DestinationCidrBlock: "0.0.0.0/0"
    NatGatewayId: !Ref MyNatGatewayA
    RouteTableId: !Ref MyRouteTableA
MyNatGatewayEIPB:
  Type: AWS::EC2::EIP
  Properties:
    Domain: vpc
MyNatGatewayB:
  Type: AWS::EC2::NatGateway
  Properties:
    AllocationId: !GetAtt MyNatGatewayEIPB.AllocationId
    ConnectivityType: public
    SubnetId: !Ref MyPrivateSubnetB
MyNatGatewayRouteB:
  Type: AWS::EC2::Route
  Properties:
    DestinationCidrBlock: "0.0.0.0/0"
    NatGatewayId: !Ref MyNatGatewayB
    RouteTableId: !Ref MyRouteTableB
MyRouteTableAssociationA:
  Type: AWS::EC2::SubnetRouteTableAssociation
  Properties:
    RouteTableId: !Ref MyRouteTableA
    SubnetId: !Ref MyPrivateSubnetA
MyRouteTableAssociationB:
  Type: AWS::EC2::SubnetRouteTableAssociation
  Properties:
    RouteTableId: !Ref MyRouteTableB
    SubnetId: !Ref MyPrivateSubnetB

ECS CloudFormation配置 #

MyExecutionRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: "2012-10-17"
      Statement:
        - Effect: Allow
          Principal:
            Service: ecs-tasks.amazonaws.com
          Action: sts:AssumeRole
    ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
      - arn:aws:iam::aws:policy/CloudWatchFullAccess
    Policies:
      - PolicyName: "Logs"
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action:
                - logs:Create
                - logs:PutLogEvents
                - logs:CreateLogStream
                - logs:PutDestination
              Resource: "arn:aws:logs:*:*:*"
      - PolicyName: "GetSecrets"
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action:
                - ssm:GetParameters
                - kms:Decrypt
                - secretsmanager:GetSecretValue
              Resource:
                - !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/*"
                - !Sub "arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:*"
                - !Sub "arn:aws:kms:${AWS::Region}:${AWS::AccountId}:key/*"
MyECSLogsGroup:
  Type: AWS::Logs::LogGroup
  Properties:
    LogGroupName: "/ecs/website"
    RetentionInDays: 90
MyECSCluster:
  Type: AWS::ECS::Cluster
  Properties:
    ClusterName: website
    Tags:
      - Key: Env
        Value: production
MyECSTaskDefinition:
  Type: AWS::ECS::TaskDefinition
  Properties:
    ContainerDefinitions:
      - Command:
          - /app/docker-entry.sh
        LogConfiguration:
          LogDriver: awslogs
          Options:
            awslogs-region: !Sub "${AWS::Region}"
            awslogs-group: !Ref MyECSLogsGroup
            awslogs-stream-prefix: qjr
        Environment:
          - Name: REDIS_HOST
            Value: !Ref RedisHost
          - Name: REDIS_PORT
            Value: !Ref RedisPort
        Essential: true
        Image: !Sub "${RepositoryUri}:latest"
        Memory: !!int 1024
        Name: website
        Cpu: !!int 512
        PortMappings:
          - AppProtocol: http
            ContainerPort: !!int 4200
            HostPort: !!int 4200
            Protocol: tcp
        Secrets:
          - Name: DATABASE_URL
            ValueFrom: !Ref DatabaseUrlArn
          - Name: GH_TOKEN
            ValueFrom: !Ref GHTokenArn
          - Name: JWT_SECRET
            ValueFrom: !Ref JWTSecretArn
          - Name: SECRET_KEY
            ValueFrom: !Ref SecretKeyArn
          - Name: YUBIKEY_CLIENT_ID
            ValueFrom: !Ref YubikeyClientIdArn
          - Name: YUBIKEY_CLIENT_SECRET
            ValueFrom: !Ref YubikeyClientSecretArn
        User: node:node
    Cpu: !!int 512
    ExecutionRoleArn: !GetAtt MyExecutionRole.Arn
    Memory: !!int 1024
    NetworkMode: awsvpc
    RequiresCompatibilities:
      - FARGATE
    RuntimePlatform:
      CpuArchitecture: X86_64
      OperatingSystemFamily: LINUX
    TaskRoleArn: !GetAtt MyTaskRole.Arn
MyECSService:
  DependsOn:
    - MyECSTaskDefinition
    - MyTargetGroup
    - MyLoadBalancer
    - MyLoadBalancerListener
    - MyECSIngressSecurityGroup
    - MyECSEgressSecurityGroup
  Type: AWS::ECS::Service
  Properties:
    Cluster: !Ref MyECSCluster
    DeploymentConfiguration:
      MaximumPercent: !!int 100
      MinimumHealthyPercent: !!int 0
    DeploymentController:
      Type: ECS
    DesiredCount: !!int 1
    LaunchType: FARGATE
    LoadBalancers:
      - ContainerName: website
        ContainerPort: !!int 4200
        TargetGroupArn: !Ref MyTargetGroup
    NetworkConfiguration:
      AwsvpcConfiguration:
        AssignPublicIp: DISABLED
        SecurityGroups:
          - !GetAtt MyECSEgressSecurityGroup.GroupId
          - !GetAtt MyECSIngressSecurityGroup.GroupId
        Subnets: !Ref PrivateSubnets
    PlatformVersion: LATEST
    SchedulingStrategy: REPLICA
    ServiceName: website
    TaskDefinition: !Ref MyECSTaskDefinition
MyECSIngressSecurityGroup:
  Type: AWS::EC2::SecurityGroup
  Properties:
    VpcId: !Ref VpcId
    GroupDescription: Inbound Traffic
    SecurityGroupIngress:
      - CidrIp: 0.0.0.0/0
        IpProtocol: tcp
        FromPort: !!int 4200
        ToPort: !!int 4200
MyECSEgressSecurityGroup:
  Type: AWS::EC2::SecurityGroup
  Properties:
    VpcId: !Ref VpcId
    GroupDescription: Outbound traffic
    SecurityGroupEgress:
      - CidrIp: 0.0.0.0/0
        IpProtocol: -1

排查与修复步骤 #

1. 修正NAT网关部署位置 #

NAT网关必须部署在公有子网才能访问互联网，当前配置将其放在了私有子网，导致NAT网关本身无法连接公网，进而私有子网内的Fargate任务无法通过它访问AWS服务。

• 修改MyNatGatewayA的SubnetId为MyPublicSubnetA
• 修改MyNatGatewayB的SubnetId为MyPublicSubnetB

2. 配置公有子网路由表 #

当前公有子网未关联带互联网路由的自定义路由表，默认使用的MyDefaultRouteTable没有指向互联网网关的默认路由：

• 创建公有子网专用路由表，添加0.0.0.0/0指向MyInternetGateway的路由
• 将MyPublicSubnetA和MyPublicSubnetB关联到该路由表

3. 验证DNS解析能力 #

虽然VPC已开启DNS支持，但需确认私有子网内的任务能解析AWS服务域名：

• 启动一个测试Fargate任务（使用amazonlinux镜像），执行nslookup ssm.us-east-2.amazonaws.com验证域名解析是否正常

4. 补充检查IAM权限细节 #

• 确认AmazonECSTaskExecutionRolePolicy托管策略已正确关联（该策略默认包含访问SSM、SecretsManager的必要权限）
• 检查密钥/参数的ARN是否正确，尤其是SecretsManager自动生成的密钥ARN是否包含末尾的随机后缀

5. 确认安全组关联 #

检查MyECSEgressSecurityGroup是否正确关联到任务所在的私有子网，确保出站规则生效

内容的提问来源于stack exchange，提问作者Joseph Quinn