如何修改Cloud Workflows代码以执行单个Dataform Sqlx转换?
问题:如何通过Cloud Workflows仅执行单个Dataform转换任务?
我被指派开展一项POC任务,使用Google Cloud Workflows与Cloud Scheduler执行Dataform仓库中的转换任务。按照官方文档提供的YAML代码操作时,代码会执行仓库内所有.sqlx文件,想知道如何修改代码使其仅执行指定的单个Dataform转换任务。
原代码如下:
main: steps: - init: assign: - repository: projects/PROJECT_ID/locations/REPOSITORY_LOCATION/repositories/REPOSITORY_ID - createCompilationResult: call: http.post args: url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/compilationResults"} auth: type: OAuth2 body: gitCommitish: GIT_COMMITISH result: compilationResult - createWorkflowInvocation: call: http.post args: url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/workflowInvocations"} auth: type: OAuth2 body: compilationResult: ${compilationResult.body.name} result: workflowInvocation - complete: return: ${workflowInvocation.body.name}
解决方案
要实现仅执行单个指定的Dataform转换任务,需在createWorkflowInvocation步骤的请求体中添加invocationConfig配置,明确指定要运行的任务名称。修改后的完整代码如下:
main: steps: - init: assign: - repository: projects/PROJECT_ID/locations/REPOSITORY_LOCATION/repositories/REPOSITORY_ID - targetTask: "YOUR_TARGET_TASK_NAME" # 替换为你要执行的单个任务名称 - createCompilationResult: call: http.post args: url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/compilationResults"} auth: type: OAuth2 body: gitCommitish: GIT_COMMITISH result: compilationResult - createWorkflowInvocation: call: http.post args: url: ${"https://dataform.googleapis.com/v1beta1/" + repository + "/workflowInvocations"} auth: type: OAuth2 body: compilationResult: ${compilationResult.body.name} # 新增配置:指定仅执行目标任务 invocationConfig: includedTargets: - name: ${targetTask} result: workflowInvocation - complete: return: ${workflowInvocation.body.name}
修改说明:
- 在
init步骤中新增targetTask变量,存储你要执行的单个任务名称(对应Dataform中.sqlx文件里定义的任务ID)。 - 在
createWorkflowInvocation的body中添加invocationConfig.includedTargets数组,通过name字段锁定具体任务,这样Workflow会仅执行该任务,而非所有.sqlx文件。 - 请替换代码中的
PROJECT_ID、REPOSITORY_LOCATION、REPOSITORY_ID、GIT_COMMITISH以及YOUR_TARGET_TASK_NAME为你的实际环境信息。
内容的提问来源于stack exchange,提问作者mj8701
相关产品推荐
相关产品推荐

