AWS目标组实例健康检查失败，ELB返回502错误求助

问题描述 #

在Udacity的AWS项目中部署包含弹性负载均衡器、自动扩展组、监听器、监听器规则、安全组和目标组的基础设施后，出现以下问题：

• 目标组内实例健康检查始终显示不健康
• 调用负载均衡DNS地址或实例IP均返回502 Bad Gateway
• 无法通过负载均衡URL看到Apache Tomcat启动页面

部署前已创建VPC栈，以下是对应的CloudFormation代码：

VPC栈代码 #

Parameters:
    EnvironmentName:
        Description: An environment name that will be prefixed to resource names
        Type: String

    VpcCIDR:
        Description: Please enter the IP range (CIDR notatio) for this VPC
        Type: String
        Default: 10.0.0.0/16

    PublicSubnet1CIDR:
        Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone
        Type: String
        Default: 10.0.0.0/24

    PublicSubnet2CIDR:
        Description: Please enter the IP range (CIDR notation) for the public subnet in the second Availability Zone
        Type: String
        Default: 10.0.1.0/24

    PrivateSubnet1CIDR:
        Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone
        Type: String
        Default: 10.0.2.0/24
    PrivateSubnet2CIDR:
        Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone
        Type: String
        Default: 10.0.3.0/24
Resources:
    VPC:
        Type: AWS::EC2::VPC
        Properties:
            CidrBlock: !Ref VpcCIDR
            EnableDnsHostnames: true
            Tags:
                - Key: Name
                  Value: !Ref EnvironmentName
    InternetGateway:
        Type: AWS::EC2::InternetGateway
        Properties:
            Tags:
                - Key: Name
                  Value: !Ref EnvironmentName
    InternetGatewayAttachment:
        Type: AWS::EC2::VPCGatewayAttachment
        Properties:
            InternetGatewayId: !Ref InternetGateway
            VpcId: !Ref VPC

    PublicSubnet1:
        Type: AWS::EC2::Subnet
        Properties:
            VpcId: !Ref VPC
            AvailabilityZone: !Select [0, !GetAZs ""]
            CidrBlock: !Ref PublicSubnet1CIDR
            MapPublicIpOnLaunch: true
            Tags:
                - Key: Name
                  Value: !Sub ${EnvironmentName} Public Subnet (AZ1)

    PublicSubnet2:
        Type: AWS::EC2::Subnet
        Properties:
            VpcId: !Ref VPC
            AvailabilityZone: !Select [1, !GetAZs ""]
            CidrBlock: !Ref PublicSubnet2CIDR
            MapPublicIpOnLaunch: true
            Tags:
                - Key: Name
                  Value: !Sub ${EnvironmentName} Public Subnet (AZ2)

    PrivateSubnet1:
        Type: AWS::EC2::Subnet
        Properties:
            VpcId: !Ref VPC
            AvailabilityZone: !Select [0, !GetAZs ""]
            CidrBlock: !Ref PrivateSubnet1CIDR
            MapPublicIpOnLaunch: false
            Tags:
                - Key: Name
                  Value: !Sub ${EnvironmentName} Private Subnet (AZ1)

    PrivateSubnet2:
        Type: AWS::EC2::Subnet
        Properties:
            VpcId: !Ref VPC
            AvailabilityZone: !Select [1, !GetAZs ""]
            CidrBlock: !Ref PrivateSubnet2CIDR
            MapPublicIpOnLaunch: false
            Tags:
                - Key: NameAW
                  Value: !Sub ${EnvironmentName} Private Subnet (AZ2)

    NatGateway1EIP:
        Type: AWS::EC2::EIP
        DependsOn: InternetGatewayAttachment
        Properties:
            Domain: vpc

    NatGateway2EIP:
        Type: AWS::EC2::EIP
        DependsOn: InternetGatewayAttachment
        Properties:
            Domain: vpc

    NatGateway1:
        Type: AWS::EC2::NatGateway
        Properties:
            AllocationId: !GetAtt NatGateway1EIP.AllocationId
            SubnetId: !Ref PublicSubnet1

    NatGateway2:
        Type: AWS::EC2::NatGateway
        Properties:
            AllocationId: !GetAtt NatGateway2EIP.AllocationId
            SubnetId: !Ref PublicSubnet2

    PublicRouteTable:
        Type: AWS::EC2::RouteTable
        Properties: 
            VpcId: !Ref VPC
            Tags: 
                - Key: Name 
                  Value: !Sub ${EnvironmentName} Public Routes

    DefaultPublicRoute: 
        Type: AWS::EC2::Route
        DependsOn: InternetGatewayAttachment
        Properties: 
            RouteTableId: !Ref PublicRouteTable
            DestinationCidrBlock: 0.0.0.0/0
            GatewayId: !Ref InternetGateway

    PublicSubnet1RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
            RouteTableId: !Ref PublicRouteTable
            SubnetId: !Ref PublicSubnet1

    PublicSubnet2RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
            RouteTableId: !Ref PublicRouteTable
            SubnetId: !Ref PublicSubnet2

    PrivateRouteTable1:
        Type: AWS::EC2::RouteTable
        Properties: 
            VpcId: !Ref VPC
            Tags: 
                - Key: Name 
                  Value: !Sub ${EnvironmentName} Private Routes (AZ1)

    DefaultPrivateRoute1:
        Type: AWS::EC2::Route
        Properties:
            RouteTableId: !Ref PrivateRouteTable1
            DestinationCidrBlock: 0.0.0.0/0
            NatGatewayId: !Ref NatGateway1

    PrivateSubnet1RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
            RouteTableId: !Ref PrivateRouteTable1
            SubnetId: !Ref PrivateSubnet1

    PrivateRouteTable2:
        Type: AWS::EC2::RouteTable
        Properties: 
            VpcId: !Ref VPC
            Tags: 
                - Key: Name 
                  Value: !Sub ${EnvironmentName} Private Routes (AZ2)

    DefaultPrivateRoute2:
        Type: AWS::EC2::Route
        Properties:
            RouteTableId: !Ref PrivateRouteTable2
            DestinationCidrBlock: 0.0.0.0/0
            NatGatewayId: !Ref NatGateway2

    PrivateSubnet2RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
            RouteTableId: !Ref PrivateRouteTable2
            SubnetId: !Ref PrivateSubnet2

Outputs:
    VPC:
        Description: A reference to the created VPC
        Value: !Ref VPC
        Export:
            Name: !Sub ${EnvironmentName}-VPCID

    PublicSubnets:
        Description: A list of the public subnets
        Value: !Join [",", [!Ref PublicSubnet1, !Ref PublicSubnet2]]
        Export:
            Name: !Sub ${EnvironmentName}-PUB-NETS

    PrivateSubnets:
        Description: A list of the private subnets
        Value: !Join [",", [!Ref PrivateSubnet1, !Ref PrivateSubnet2]]
        Export:
            Name: !Sub ${EnvironmentName}-PRIV-NETS

    PublicSubnet1:
        Description: A reference to the public subnet in the 1st Availability Zone
        Value: !Ref PublicSubnet1
        Export:
            Name: !Sub ${EnvironmentName}-PUB1-SN

    PublicSubnet2:
        Description: A reference to the public subnet in the 2nd Availability Zone
        Value: !Ref PublicSubnet2
        Export:
            Name: !Sub ${EnvironmentName}-PUB2-SN

    PrivateSubnet1:
        Description: A reference to the private subnet in the 1st Availability Zone
        Value: !Ref PrivateSubnet1
        Export:
            Name: !Sub ${EnvironmentName}-PRI1-SN

    PrivateSubnet2:
        Description: A reference to the private subnet in the 2nd Availability Zone
        Value: !Ref PrivateSubnet2
        Export:
            Name: !Sub ${EnvironmentName}-PRI2-SN

应用层栈代码 #

Description: >
  xxx / Udacity 2023

Parameters:
  EnvironmentName:
    Description: An environment name that will be prefixed to resource names
    Type: String
  myLaunchTemplateVersionNumber:
    Type: String
    Default: 1

Resources:
  LBSecGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow http to our load balancer
      VpcId: 
        Fn::ImportValue:
          !Sub "${EnvironmentName}-VPCID"
      SecurityGroupIngress: 
      - IpProtocol: tcp
        FromPort: 80
        ToPort: 80
        CidrIp: 0.0.0.0/0 
      SecurityGroupEgress:
      - IpProtocol: tcp
        FromPort: 80
        ToPort: 80
        CidrIp: 0.0.0.0/0
  WebServerSecGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow http to our hosts and SSH from local only
      VpcId:
        Fn::ImportValue:
          !Sub "${EnvironmentName}-VPCID"
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: 8080
        ToPort: 8080
        CidrIp: 0.0.0.0/0
      - IpProtocol: tcp 
        FromPort: 22
        ToPort: 22
        CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
      - IpProtocol: tcp
        FromPort: 0
        ToPort: 65535
        CidrIp: 0.0.0.0/0
  myWebAppLaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      LaunchTemplateData:
        UserData:
          Fn::Base64: !Sub |
            #!/bin/bash
            apt-get update -y
            apt-get install apache2 -y
            systemctl start apache2.service
            cd /var/www/html
            echo "Udacity Demo Web Server Up and Running!" > index.html
        ImageId: ami-0a261c0e5f51090b1 
        KeyName: mykey 
        SecurityGroupIds:    
          - sg-01ad772aba0f98d98
        InstanceType: t3.medium
        BlockDeviceMappings:
          - DeviceName: "/dev/sdk"
            Ebs:
              VolumeSize: '10'
  WebAppGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      VPCZoneIdentifier:
        - Fn::ImportValue: !Sub "${EnvironmentName}-PUB-NETS"
      LaunchTemplate:
        LaunchTemplateId: !Ref myWebAppLaunchTemplate
        Version: !Ref myLaunchTemplateVersionNumber
      MinSize: "3"
      MaxSize: "5"
      TargetGroupARNs: 
      - Ref: WebAppTargetGroup 
  WebAppLB:  
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer 
    Properties:
      Subnets: 
      - Fn::ImportValue: !Sub "${EnvironmentName}-PUB1-SN"
      - Fn::ImportValue: !Sub "${EnvironmentName}-PUB2-SN"
      SecurityGroups:
      - Ref: LBSecGroup
  Listener: 
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
      - Type: forward
        TargetGroupArn:
          Ref: WebAppTargetGroup
      LoadBalancerArn:
        Ref: WebAppLB
      Port: '80' 
      Protocol: HTTP
  ALBListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Actions:
      - Type: forward
        TargetGroupArn: !Ref 'WebAppTargetGroup'
      Conditions:
      - Field: path-pattern
        Values: [/] 
      ListenerArn: !Ref 'Listener'
      Priority: 1
  WebAppTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties: 
      HealthCheckIntervalSeconds: 30 
      HealthCheckPath: / 
      HealthCheckProtocol: HTTP 
      HealthCheckTimeoutSeconds: 15
      HealthyThresholdCount: 2 
      UnhealthyThresholdCount: 5 
      Port: 8080 
      Protocol: HTTP
      VpcId:
        Fn::ImportValue:
          Fn::Sub: "${EnvironmentName}-VPCID"

问题排查与修复方案 #

1. 负载均衡安全组出站规则限制 #

问题：LBSecGroup的出站规则仅允许TCP 80端口流量，但目标组健康检查和流量转发都使用8080端口，导致负载均衡无法向实例发送健康检查请求，也无法转发用户流量到实例的8080端口。

修复：修改LBSecGroup的SecurityGroupEgress规则，允许所有TCP端口：

SecurityGroupEgress:
- IpProtocol: tcp
  FromPort: 0
  ToPort: 65535
  CidrIp: 0.0.0.0/0

2. 启动模板安全组硬编码 #

问题：myWebAppLaunchTemplate中的SecurityGroupIds使用了固定的安全组ID，而非引用当前栈中创建的WebServerSecGroup，可能导致实例使用错误的安全组，无法接收负载均衡的流量。

修复：将硬编码的安全组ID替换为引用：

SecurityGroupIds:    
  - !Ref WebServerSecGroup

3. 实例服务端口与目标组端口不匹配 #

问题：用户数据中安装的是Apache2（默认监听80端口），但目标组配置的是检查和转发到8080端口，导致实例在8080端口无服务运行，健康检查失败；同时用户需求是看到Tomcat启动页面，但当前脚本未安装Tomcat。

修复：选择以下两种方案之一：

• 方案一：适配Apache2端口：修改目标组的端口为80：

  WebAppTargetGroup:
    Properties: 
      Port: 80 
  

• 方案二：安装Tomcat服务：修改UserData脚本，安装并启动Tomcat（默认监听8080端口）：

  UserData:
    Fn::Base64: !Sub |
      #!/bin/bash
      apt-get update -y
      apt-get install openjdk-11-jdk -y
      wget https://dlcdn.apache.org/tomcat/tomcat-9/v9.0.85/bin/apache-tomcat-9.0.85.tar.gz
      tar xzf apache-tomcat-9.0.85.tar.gz
      mv apache-tomcat-9.0.85 /opt/tomcat
      chmod +x /opt/tomcat/bin/startup.sh
      /opt/tomcat/bin/startup.sh
  

4. VPC栈中子网标签错误（非功能性问题） #

问题：PrivateSubnet2的标签Key为NameAW，应为Name，属于配置规范问题。

修复：修改标签Key：

Tags:
  - Key: Name
    Value: !Sub ${EnvironmentName} Private Subnet (AZ2)

内容的提问来源于stack exchange，提问作者ugurkocak1980