AWS Glue Job CloudFormation配置参数未生效问题排查

如下CloudFormation配置部署后未达到预期效果：使用该模板部署AWS Glue Job时，IAM Role、Type、Language三个参数无法被正确赋值，手动补充配置这三个参数后Glue Job即可正常运行。

• IAM Role
• Type
• Language

问题模板 #

Description: "AWS Glue Job Test"
Resources:
  MyJobRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: "Allow"
            Principal:
              Service:
                - "glue.amazonaws.com"
            Action:
              - "sts:AssumeRole"
      Path: "/"
      Policies:
        - PolicyName: "root"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: "Allow"
                Action: "*"
                Resource: "*"
  MyJob:
    Type: AWS::Glue::Job
    Properties:
      Name: cf-job1
      Command:
        Name: test-etl1
        ScriptLocation: "s3://project_bucket/releases/latest/mixpanel_job.py"
        PythonVersion: "3"
      Description: "Testing setup config"
      ExecutionProperty:
        MaxConcurrentRuns: 2
      MaxRetries: 2
      GlueVersion: "3.0"
      WorkerType: "G.1X"
      NumberOfWorkers: 2
      Timeout: 2880
      DefaultArguments:
        "--class": "GlueApp"
        "--enable-continuous-cloudwatch-log": "true"
        "--enable-job-insights": "true"
        "--enable-metrics": "true"
        "--enable-spark-ui": "true"
        "--extra-jars": "s3://project_bucket/releases/latest/jars/delta-core_2.12-1.0.1.jar"
        "--extra-py-files": "s3://project_bucket/releases/latest/lib.zip"
        "--job-bookmark-option": "job-bookmark-disable"
        "--job-language": "python"
        "--spark-event-logs-path": "s3://project_bucket/logs/"
      Role: !Ref MyJobRole

（附：堆栈部署结果截图） #

三个参数赋值失败的核心原因是模板中AWS::Glue::Job资源的Command.Name字段配置不符合AWS官方规范：

1. Type参数识别失败：Glue Job的类型完全由Command.Name的固定取值决定，模板中填写的自定义值test-etl1不是AWS认可的合法枚举值，服务端无法识别作业类型
2. Language参数识别失败：虽然DefaultArguments中配置了--job-language: python，但非法的Command配置会触发资源校验逻辑异常，导致该参数被服务端忽略
3. IAM Role关联失败：Role字段的关联逻辑依赖Glue Job资源的合法校验通过，Command配置非法时，Role的Ref关联不会被正常写入作业配置

将MyJob资源下Command块的Name字段替换为AWS规定的合法取值即可。针对当前使用Glue 3.0、Python 3的Spark ETL作业场景，修改后的Command块配置如下：

Command:
  Name: glueetl
  ScriptLocation: "s3://project_bucket/releases/latest/mixpanel_job.py"
  PythonVersion: "3"

Glue Command.Name 固定取值对应关系：

• glueetl：Spark ETL作业（支持Python/Scala，匹配当前使用场景）
• pythonshell：Python Shell类型作业
• gluestreaming：Spark流处理作业

修改完成后重新部署CloudFormation堆栈，三个参数会被自动正确赋值，无需手动调整。

内容的提问来源于stack exchange，提问作者Nick