GCP Batch结合Workflows调用实例模板失败求助

我尝试通过Batch API调用Compute Engine实例模板，该模板单独作为普通Compute Engine实例运行完全正常，但通过Batch结合Workflows调用时，实例会在10-15分钟后被终止，Batch任务状态全程卡在SCHEDULED，失败时实例日志显示执行了compute.instances.delete操作。

Workflows代码 #

main:
  params: [args]
  steps:
    - init:
        assign:
          - projectId: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
          - region: "us-central1"
          - batchApi: "batch.googleapis.com/v1"
          - batchApiUrl: ${"https://" + batchApi + "/projects/" + projectId + "/locations/" + region + "/jobs"}
          - jobId: ${"fm-" + string(int(sys.now()))}
    - createAndRunBatchJob:
        call: http.post
        args:
          url: ${batchApiUrl}
          query:
            job_id: ${jobId}
          headers:
            Content-Type: application/json
          auth:
            type: OAuth2
          body:
            taskGroups:
              taskSpec:
                runnables:
                  - script:
                      text: "echo Hello world! This is task ${BATCH_TASK_INDEX}."
              taskCount: 1
              parallelism: 1
            allocationPolicy:
                instances:
                    - instanceTemplate: "updateservice-11"
            logsPolicy:
              destination: CLOUD_LOGGING
        result: createAndRunBatchJobResponse
    - getJob:
        call: http.get
        args:
          url: ${batchApiUrl + "/" + jobId}
          auth:
            type: OAuth2
        result: getJobResult
    - logState:
        call: sys.log
        args:
          data: ${"Current job state " + getJobResult.body.status.state}
    - checkState:
        switch:
          - condition: ${getJobResult.body.status.state == "SUCCEEDED"}
            next: returnResult
          - condition: ${getJobResult.body.status.state == "FAILED"}
            next: failExecution
        next: sleep
    - sleep:
        call: sys.sleep
        args:
          seconds: 10
        next: getJob
    - returnResult:
        return:
          jobId: ${jobId}
    - failExecution:
        raise:
          message: ${"The underlying batch job " + jobId + " failed"}

失败时的实例日志 #

{
insertId: "1ojqzgde8yq"
logName: "projects/XXXXX/logs/cloudaudit.googleapis.com%2Factivity"
operation: {3}
protoPayload: {
@type: "type.googleapis.com/google.cloud.audit.AuditLog"
authenticationInfo: {1}
methodName: "v1.compute.instances.delete"
request: {
@type: "type.googleapis.com/compute.instances.delete"
}
requestMetadata: {1}
resourceName: "projects/XXXXXXX/zones/us-central1-a/instances/j-1aaa7f73-3865-45a7-b0ea-fea8eeaa68a2-group0-0-s97m"
serviceName: "compute.googleapis.com"
}
receiveTimestamp: "2022-10-10T20:28:36.519771048Z"
resource: {2}
severity: "NOTICE"
timestamp: "2022-10-10T20:28:36.119832Z"
}

1. 检查实例模板的启动脚本兼容性 #

如果实例模板包含自定义启动脚本，需确保：

• 脚本不会无限挂起或阻塞系统进程，否则会导致Batch代理无法完成初始化
• 脚本执行完毕后不会干扰Batch服务的正常运行

2. 验证Batch服务权限 #

确认Batch服务账号（service-<PROJECT_NUMBER>@gcp-sa-batch.iam.gserviceaccount.com）拥有以下权限：

• instanceTemplates.use：确保能读取指定的实例模板
• roles/batch.jobEditor或等价权限：保证Batch能完整管理实例生命周期

3. 调整任务超时配置并查看详细日志 #

默认10-15分钟的超时是Batch对实例初始化失败的自动清理机制，可通过以下方式排查：

• 在taskSpec中添加超时配置，延长初始化时间：

  taskSpec:
    maxRunDuration: "3600s" # 根据实际需求调整时长
    runnables:
      - script:
          text: "echo Hello world! This is task ${BATCH_TASK_INDEX}."
  

• 执行gcloud batch jobs describe <JOB_ID> --region us-central1获取任务的failureReason字段，定位具体失败原因

4. 检查网络配置 #

• 确保实例模板使用的子网允许出站流量访问batch.googleapis.com，私有VPC需配置Cloud NAT或Private Service Connect
• 检查防火墙规则是否限制了实例与Batch服务的通信

内容的提问来源于stack exchange，提问作者sonium