从Step Functions调用Glue Workflows遇CloudFormation部署错误求助
部署CloudFormation时,从Step Functions调用Glue Workflow遇到以下错误:
Resource handler returned message: "Invalid State Machine Definition: 'SCHEMA_VALIDATION_FAILED: The resource provided arn:aws:states:::glue:startWorkflowRun is not recognized. The value is not a valid resource ARN, or the resource is not available in this region. at /States/StartGlueWorkflow/Resource' (Service: AWSStepFunctions; Status Code: 400; Error Code: InvalidDefinition;..., HandlerErrorCode: InvalidRequest)
用于启动Glue Workflow的状态机任务配置如下:
"StartGlueWorkflow": { "Type": "Task", "Resource": "arn:aws:states:::glue:startWorkflowRun", "Parameters": { "Name": "glue-test" }, "End": true },
Glue Workflow“glue-test”存在于同一区域(eu-central-1),并配置了按需触发器。Step Functions状态机的IAM角色已配置Glue Workflow相关权限:
RoleStmaMain: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: !Sub 'states.${AWS::Region}.amazonaws.com' Action: 'sts:AssumeRole' Policies: - PolicyName: role-stma-policy PolicyDocument: Statement: ... ... - Effect: Allow Action: - "glue:StartWorkflowRun" - "glue:GetWorkflowRun" - "glue:GetWorkflow" - "glue:BatchGetJobs" - "glue:BatchGetCrawlers" Resource: - "*"
备注:Step Functions为标准类型,区域为eu-central-1。
疑问:该ARN(arn:aws:states:::glue:startWorkflowRun)是否正确?Step Functions是否支持调用Glue Workflows?多数示例仅展示Step Functions与Glue Job的集成,未涉及Glue Workflow。
你使用的ARN不正确
Step Functions没有arn:aws:states:::glue:startWorkflowRun这种预定义的任务资源ARN,只有针对Glue Job的预集成资源(比如arn:aws:states:::glue:startJobRun),所以会报资源不被识别的错误。Step Functions支持调用Glue Workflow,但需用AWS SDK集成方式
要启动Glue Workflow,需要使用Step Functions的AWS SDK服务集成类型,对应的ARN是arn:aws:states:::aws-sdk:glue:startWorkflowRun。
修正后的状态机任务配置如下:
"StartGlueWorkflow": { "Type": "Task", "Resource": "arn:aws:states:::aws-sdk:glue:startWorkflowRun", "Parameters": { "Name": "glue-test" }, "End": true }
补充说明
- 这种方式会直接调用Glue的
StartWorkflowRunAPI,你当前配置的IAM角色已经包含了glue:StartWorkflowRun等必要权限,无需额外调整。 - 由于Glue Workflow的Step Functions集成示例较少,容易混淆预集成资源和SDK集成的用法,后者是调用Glue未提供预集成支持的API的通用方式。
内容的提问来源于stack exchange,提问作者dotmindlabs

