Github Actions执行EC2重启后未完成验证即退出问题排查

问题描述 #

我用Github Actions实现了一个工作流，通过SSH连接EC2实例，执行yum更新、重启操作，并验证重启后系统是否正常运行。更新环节运行正常，但验证环节失败，报错信息如下：

out: No updates available
out: Rebooting the machine...
2023/02/22 20:47:14 wait: remote command exited without exit status or exit signal

原工作流步骤代码：

steps:
    - name: SSH into VMs
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        username: ${{ secrets.USERNAME }}
        password: ${{ secrets.PASSWORD }}
        script: |
          if sudo yum check-update; then
            sudo yum update -y
          else
            echo "No updates available"
          fi
          echo "Rebooting the machine..."
          sudo reboot
          sleep 600
    - name: Wait for VMs to come back
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        username: ${{ secrets.USERNAME }}
        password: ${{ secrets.PASSWORD }}
        script: |
          echo "Running vm to comeback"
          until ping -c1 ${{ secrets.HOST }}; do sleep 1; done
          if ssh -o StrictHostKeyChecking=no ${{ secrets.USERNAME }}@${{ secrets.HOST }} 'exit'; then
            echo "Success: VM is back up and running"
          else
            echo "Error: Failed to connect to the VM"
            exit 1
          fi

问题原因 #

1. 第一个SSH步骤中，执行sudo reboot会直接终止远程SSH会话，后续的sleep 600根本不会执行，导致ssh-action捕获到异常退出状态，触发报错。
2. 第二个SSH步骤在远程实例内部执行ping和嵌套ssh连接自身属于冗余操作，且无法精准判断SSH服务是否完全就绪。

修复后的工作流代码 #

steps:
    - name: SSH into VMs and perform updates/reboot
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        username: ${{ secrets.USERNAME }}
        password: ${{ secrets.PASSWORD }}
        script: |
          if sudo yum check-update; then
            sudo yum update -y
          else
            echo "No updates available"
          fi
          echo "Rebooting the machine..."
          # 将reboot放到后台执行，立即正常退出会话，避免异常报错
          sudo reboot &
          exit 0
    - name: Wait for VM to recover and verify connectivity
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        username: ${{ secrets.USERNAME }}
        password: ${{ secrets.PASSWORD }}
        # 配置重试次数和超时时间，自动等待实例恢复SSH服务
        retry: 10
        timeout: 30
        script: |
          echo "VM is back online and SSH connection is successful"

修改说明 #

• 第一个步骤：通过&将sudo reboot放到后台执行，随后立即exit 0让SSH会话正常结束，避免触发异常退出报错。
• 第二个步骤：利用ssh-action自带的retry和timeout参数实现自动重试连接，无需手动编写等待逻辑，更精准可靠地验证实例是否恢复正常。

内容的提问来源于stack exchange，提问作者bisamov