通过GCP Workflows编排Dataform工作流时遇部署错误

使用GCP Workflows和Cloud Scheduler调度Dataform执行，目标是仅运行带有hourly标签的模型，编写的YAML配置在部署时触发以下错误：

无法部署工作流：构建失败：createCompilationResult步骤出错：评估'body'键的属性时出错：评估'compilationResult'键的属性时出错：符号'compilationResult.body.invocationConfig'未定义，符号'compilationResult.body'未定义，且符号'compilationResult'既不是变量也不是子工作流名称（代码：3）

原配置代码如下：

main:
    steps:
    - init:
        assign:
        - repository: projects/XXXXX/locations/europe-west4/repositories/XXXXX
    - createCompilationResult:
        call: http.post
        args:
            url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/compilationResults"}
            auth:
                type: OAuth2
            body:
                compilationResult: ${compilationResult.body.name}                
                invocationConfig:
                    includedTags:
                    - hourly
                    transitiveDependenciesIncluded: true
        result: compilationResult
    - createWorkflowInvocation:
        call: http.post
        args:
            url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/workflowInvocations"}
            auth:
                type: OAuth2
            body:
                compilationResult: ${compilationResult.body.name}
        result: workflowInvocation
    - complete:
        return: ${workflowInvocation.body.name}

错误原因 #

1. createCompilationResult步骤的body参数错误：

   • 创建编译结果的请求不需要compilationResult字段，且此时compilationResult变量尚未定义（该变量要等当前步骤执行完成后才会赋值），直接引用${compilationResult.body.name}会触发符号未定义错误。
   • invocationConfig是工作流调用的配置参数，不属于编译结果创建请求的范畴，不应放在createCompilationResult步骤中。

2. 配置逻辑错位：
   筛选标签的invocationConfig应该放在创建工作流调用的createWorkflowInvocation步骤中，用于指定本次执行的范围。

修正后的配置 #

main:
    steps:
    - init:
        assign:
        - repository: projects/XXXXX/locations/europe-west4/repositories/XXXXX
    - createCompilationResult:
        call: http.post
        args:
            url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/compilationResults"}
            auth:
                type: OAuth2
        result: compilationResult
    - createWorkflowInvocation:
        call: http.post
        args:
            url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/workflowInvocations"}
            auth:
                type: OAuth2
            body:
                compilationResult: ${compilationResult.body.name}
                invocationConfig:
                    includedTags:
                    - hourly
                    transitiveDependenciesIncluded: true
        result: workflowInvocation
    - complete:
        return: ${workflowInvocation.body.name}

内容的提问来源于stack exchange，提问作者alek6dj