You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

如何在AWS ECS服务启动失败时让部署任务失败退出

解决AWS ECS部署失败时Azure流水线不告警且自动回滚的问题

1. 修正部署断路器参数格式

你之前使用的参数格式可能存在解析问题,AWS CLI对复杂配置更适配JSON字符串写法,替换成以下命令即可启用断路器且禁止自动回滚:

aws ecs update-service \
  --region "myRegion" \
  --force-new-deployment \
  --service myAPI \
  --cluster arn:aws:ecs:xxxxx:cluster/myClusterName \
  --deployment-configuration '{"deploymentCircuitBreaker": {"enable": true, "rollback": false}}'

2. 部署后轮询检查任务状态(核心步骤)

aws ecs update-service是异步执行的,命令返回成功不代表服务启动成功。需要在部署后添加轮询逻辑,一旦检测到任务启动失败就让流水线以exit 1终止。

以下是可直接复用的bash脚本,可嵌入到Azure Pipelines的脚本任务中:

#!/bin/bash

REGION="myRegion"
SERVICE_NAME="myAPI"
CLUSTER_ARN="arn:aws:ecs:xxxxx:cluster/myClusterName"
MAX_RETRIES=10
RETRY_INTERVAL=30

# 获取本次部署的最新部署ID
get_latest_deployment() {
  aws ecs describe-services --region $REGION --cluster $CLUSTER_ARN --services $SERVICE_NAME | \
  jq -r '.services[0].deployments | sort_by(.createdAt) | last.id'
}

DEPLOYMENT_ID=$(get_latest_deployment)
echo "监控部署ID: $DEPLOYMENT_ID"

for ((i=0; i<MAX_RETRIES; i++)); do
  # 获取当前部署状态与失败任务数
  DEPLOYMENT_STATUS=$(aws ecs describe-services --region $REGION --cluster $CLUSTER_ARN --services $SERVICE_NAME | \
  jq -r ".services[0].deployments[] | select(.id == \"$DEPLOYMENT_ID\") | .status")
  
  FAILED_TASKS=$(aws ecs describe-services --region $REGION --cluster $CLUSTER_ARN --services $SERVICE_NAME | \
  jq -r ".services[0].deployments[] | select(.id == \"$DEPLOYMENT_ID\") | .failedTasksCount")

  echo "当前部署状态: $DEPLOYMENT_STATUS,失败任务数: $FAILED_TASKS"

  if [ "$DEPLOYMENT_STATUS" = "PRIMARY" ] && [ "$FAILED_TASKS" -eq 0 ]; then
    echo "部署成功,新服务已正常运行"
    exit 0
  fi

  if [ "$FAILED_TASKS" -gt 0 ]; then
    echo "部署失败,存在启动失败的任务"
    exit 1
  fi

  if [ "$DEPLOYMENT_STATUS" = "FAILED" ]; then
    echo "部署状态标记为失败"
    exit 1
  fi

  echo "等待$RETRY_INTERVAL秒后再次检查..."
  sleep $RETRY_INTERVAL
done

echo "超过最大重试次数,部署未完成"
exit 1

脚本说明:

  • 先获取最新部署ID,确保监控的是本次触发的部署
  • 循环检查部署状态与失败任务数,最多重试10次、每次间隔30秒(可根据服务启动时长调整)
  • 若部署成为主服务且无失败任务则判定成功;出现失败任务或部署状态为FAILED时,直接以exit 1终止流水线
  • 超时未完成也判定为失败

3. Azure Pipelines集成示例

在你的azure-pipelines.yml中添加脚本任务,整合部署与状态检查逻辑:

- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      # 执行ECS部署命令
      aws ecs update-service \
        --region "myRegion" \
        --force-new-deployment \
        --service myAPI \
        --cluster arn:aws:ecs:xxxxx:cluster/myClusterName \
        --deployment-configuration '{"deploymentCircuitBreaker": {"enable": true, "rollback": false}}'

      # 执行状态检查逻辑
      REGION="myRegion"
      SERVICE_NAME="myAPI"
      CLUSTER_ARN="arn:aws:ecs:xxxxx:cluster/myClusterName"
      MAX_RETRIES=10
      RETRY_INTERVAL=30

      get_latest_deployment() {
        aws ecs describe-services --region $REGION --cluster $CLUSTER_ARN --services $SERVICE_NAME | \
        jq -r '.services[0].deployments | sort_by(.createdAt) | last.id'
      }

      DEPLOYMENT_ID=$(get_latest_deployment)
      echo "监控部署ID: $DEPLOYMENT_ID"

      for ((i=0; i<MAX_RETRIES; i++)); do
        DEPLOYMENT_STATUS=$(aws ecs describe-services --region $REGION --cluster $CLUSTER_ARN --services $SERVICE_NAME | \
        jq -r ".services[0].deployments[] | select(.id == \"$DEPLOYMENT_ID\") | .status")
        
        FAILED_TASKS=$(aws ecs describe-services --region $REGION --cluster $CLUSTER_ARN --services $SERVICE_NAME | \
        jq -r ".services[0].deployments[] | select(.id == \"$DEPLOYMENT_ID\") | .failedTasksCount")

        echo "当前部署状态: $DEPLOYMENT_STATUS,失败任务数: $FAILED_TASKS"

        if [ "$DEPLOYMENT_STATUS" = "PRIMARY" ] && [ "$FAILED_TASKS" -eq 0 ]; then
          echo "部署成功,新服务已正常运行"
          exit 0
        fi

        if [ "$FAILED_TASKS" -gt 0 ]; then
          echo "部署失败,存在启动失败的任务"
          exit 1
        fi

        if [ "$DEPLOYMENT_STATUS" = "FAILED" ]; then
          echo "部署状态标记为失败"
          exit 1
        fi

        echo "等待$RETRY_INTERVAL秒后再次检查..."
        sleep $RETRY_INTERVAL
      done

      echo "超过最大重试次数,部署未完成"
      exit 1
  env:
    AWS_ACCESS_KEY_ID: $(AWS_ACCESS_KEY_ID)
    AWS_SECRET_ACCESS_KEY: $(AWS_SECRET_ACCESS_KEY)

确保流水线已配置好AWS访问凭据,拥有ECS服务的查询权限。

内容的提问来源于stack exchange,提问作者PApostol

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.07.04 09:18:42