EC2 Auto Scaling Group实例状态上报失败，cfn-signal未执行求助

我尝试通过自定义初始化脚本启动Auto Scaling Group（ASG）内的EC2实例，但脚本执行失败导致ASG无法正常创建。服务器日志末尾显示：

Cloud-init v. 22.3.4-0ubuntu1~20.04.1 running 'modules:final' at Thu, 17 Nov 2022 21:02:38 +0000. Up 53.27 seconds.
2022-11-17 21:05:42,663 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2022-11-17 21:05:42,664 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed

我的CloudFormation模板如下：

WebLaunchConfig:
  Type: AWS::AutoScaling::LaunchConfiguration
  Properties:
    BlockDeviceMappings:
      - DeviceName: /dev/sda1
        Ebs:
          VolumeSize: 100
          VolumeType: gp2
    KeyName: devteam
    IamInstanceProfile: {Ref: WebInstanceProfile}
    ImageId: {Ref: AMI}
    InstanceType: {Ref: InstanceType}
    SecurityGroups: {Ref: WebSecurityGroups}
    UserData:
      Fn::Base64:
        !Sub |
          #!/bin/bash -ex
          full_hostname="${AWS::StackName}-$(ec2metadata --instance-id).itglue.com"
          echo $full_hostname > /etc/hostname
          hostname $full_hostname
          echo "$(ec2metadata --local-ipv4) ${!full_hostname}" >> /etc/hosts
          service rsyslog restart

          sed -i "s/environment = qa/environment = ${PuppetEnv}/" /etc/puppet/puppet.conf
          set +e

          puppet agent --enable
          service puppet stop || true
          while [[ -e /var/lib/puppet/state/agent_catalog_run.lock ]]; do
            echo "Waiting for puppet run to finish"
            sleep 10
          done

          puppet agent -t
          puppet_ret=$?
          echo $puppet_ret
          # Puppet agent error code docs under '--detailed-exitcodes'
          if [ $puppet_ret -eq 1 ] || [ $puppet_ret -eq 4 ] || [ $puppet_ret -eq 6 ]
          then
            echo "Puppet run failed!"
            exit 1
          fi
          set -e
          service puppet start

          {% if not sceptre_user_data.get('personal_env', false) %}
          service webapp stop; sleep 5 && service webapp start
          RAILS_ENV=${RailsEnv} /usr/local/rvm/bin/rvm in /var/www/webapp/current do bundle exec rake smoke_test:run
          {% endif %}

          # for hostname logging rename to take effect:
          service amazon-cloudwatch-agent restart

          test -d /var/www/webapp/current/vendor/ruby/${RubyMajorVersion}

          /usr/local/bin/cfn-signal -s true --stack ${AWS::StackName} --resource WebAutoScaleGroup --region ${AWS::Region}

排查建议 #

• 定位脚本退出点：脚本开头用了#!/bin/bash -ex，-e参数会让脚本在任意命令失败时直接退出。虽然后续临时关闭了set -e，但恢复该模式后，以下命令失败都会导致脚本终止：

  • service puppet start：puppet服务启动失败
  • service amazon-cloudwatch-agent restart：云监控代理重启失败
  • test -d /var/www/webapp/current/vendor/ruby/${RubyMajorVersion}：目标目录不存在时，测试命令返回非0，触发set -e退出

• 查看完整执行日志：登录实例查看/var/log/cloud-init-output.log，该日志会记录脚本每一步的执行输出，能直接定位到失败的命令；同时检查/var/log/cloud-init.log获取更详细的错误信息。

• 验证模板渲染结果：查看实例上/var/lib/cloud/instance/scripts/part-001文件，确认sceptre_user_data的条件判断是否生成了合法的bash语句，避免因模板渲染错误导致脚本语法失效。

• 检查Puppet返回码逻辑：确认puppet agent -t的返回码是否在判断范围内，比如Puppet返回码2（成功且有配置变更）是允许的，但如果返回其他未覆盖的码（如3），需确认后续步骤是否正常执行。

• 确认cfn-signal可用性：检查实例上是否存在/usr/local/bin/cfn-signal，部分AMI可能未预装aws-cfn-bootstrap工具，需手动安装；同时确保实例的IAM角色（WebInstanceProfile）拥有cloudformation:SignalResource权限，允许向CloudFormation发送信号。

内容的提问来源于stack exchange，提问作者Joseph Quinn