使用CloudFormation创建AWS Glue 4.0作业失败,遇版本支持报错
问题分析与解决
错误原因清晰:Glue 4.0仅支持ETL、Ray和Streaming类型的作业,你的CloudFormation模板中Command.Name字段使用了自定义名称(!Sub job-${Env}),AWS无法识别该作业类型,因此触发报错。
修正步骤
将Command.Name替换为对应作业类型的合法值:
- ETL作业:
glueetl(你的配置匹配ETL作业特征,推荐使用此值) - Ray作业:
glueray - Streaming作业:
gluestreaming
修改后的模板代码
TestJob: Type: AWS::Glue::Job Properties: Connections: Connections: - !Sub job-${Env} Command: Name: glueetl # 替换为合法的作业类型名称 ScriptLocation: !Ref JobScriptLocation ExecutionProperty: MaxConcurrentRuns: 1 MaxRetries: 3 DefaultArguments: "--enable-metrics": true "--enable-spark-ui": true "--extra-py-files": s3://aws-glue-studio-transforms-560373232017-prod-eu-central-1/gs_common.py,s3://aws-glue-studio-transforms-560373232017-prod-eu-central-1/gs_derived.py "--spark-event-logs-path": s3://test/sparkHistoryLogs/ "--enable-job-insights": false "--enable-glue-datacatalog": true "--enable-continuous-cloudwatch-log": true "--job-bookmark-option": job-bookmark-disable "--job-language": python "--TempDir": s3://test/temporary/ Name: !Sub job-${Env} Role: !Ref Role WorkerType: "G.1X" NumberOfWorkers: 5 GlueVersion: 4.0
内容的提问来源于stack exchange,提问作者Karen Kostanyan
相关产品推荐
相关产品推荐

