AWS Step Functions触发Batch作业时环境变量传递失败求助

问题分析 #

当前核心错误配置有两处：

1. 状态机中的Assign块仅用于定义任务输出结构，无法将变量传递给AWS Batch作业；
2. Terraform作业定义中存在重复的environment键（后定义的会完全覆盖前一个），且错误尝试在Batch作业定义中直接引用Step Functions的变量表达式（Batch不识别该语法）。

可行解决方案 #

方案一：通过Step Functions的ContainerOverrides传递动态环境变量 #

修改状态机的Task配置，在Arguments中添加ContainerOverrides字段，直接将Step Functions输入中的变量传递给Batch容器：

{
  "Comment": "A description of my state machine",
  "StartAt": "xxxxxx Job",
  "QueryLanguage": "JSONata",
  "States": {
    "blahblah Job": {
      "Type": "Task",
      "Resource": "arn:aws:states:::batch:submitJob.sync",
      "Arguments": {
        "JobName": "blahJob",
        "JobDefinition": "job-def-arn:69",
        "JobQueue": "job-queue",
        "ContainerOverrides": {
          "Environment": [
            {"Name": "apiCall", "Value": "$states.input.apiCall"},
            {"Name": "wid", "Value": "$states.input.wid"}
          ]
        }
      },
      "Next": "bleg Job"
    },
    "bleg Job": {
      "Type": "Task",
      "Resource": "arn:aws:states:::batch:submitJob.sync",
      "Arguments": {
        "JobDefinition": "job-arn:69",
        "JobQueue": "job-queue",
        "JobName": "blegJob",
        "ContainerOverrides": {
          "Environment": [
            {"Name": "apiCall", "Value": "$states.input.apiCall"},
            {"Name": "person", "Value": "$states.input.person"},
            {"Name": "endpoint", "Value": "$states.input.endpoint"}
          ]
        }
      },
      "End": true
    }
  }
}

说明：因状态机启用了JSONata，使用$states.input.xxx引用输入变量；若用默认JSONPath，改为$.xxx即可。

方案二：修正Terraform作业定义的环境变量配置 #

1. 移除重复的environment键：当前第一个容器中定义了两次environment，后一个会覆盖前一个，导致静态环境变量丢失、动态变量配置无效；
2. 合并静态与占位环境变量：将静态环境变量和需要动态传递的变量合并，动态变量先设为空值，后续由Step Functions的ContainerOverrides覆盖。

修改后的Terraform容器配置示例：

resource "aws_batch_job_definition" "managed" {
  for_each = var.manage_job_definitions ? var.job_definitions : {}

  name = "${var.app_name}-${var.environment}-${each.key}"
  type = "container"
  platform_capabilities = [
    "FARGATE",
  ]

  container_properties = <<CONTAINER_PROPERTIES
{
  "fargatePlatformConfiguration": {
    "platformVersion": "LATEST"
  },
  "executionRoleArn": "${aws_iam_role.ecs_task_execution_role.arn}",
  "jobRoleArn": "${local.ecs_job_role}",
  "mountPoints": ${jsonencode(var.mount_points)},
  "volumes": ${jsonencode(var.volumes)},
  "containers": [
      {
        "name": "xxxxx",
        "image": "${xxxxx}",
        "environment": ${jsonencode(concat(
          local.common_env_vars,
          each.value.environment_variables,
          [
            {"name": "apiCall", "value": ""},
            {"name": "wid", "value": ""}
          ]
        ))},
        "essential": true,
        "resourceRequirements": [
          {"type": "VCPU", "value": "0.25"},
          {"type": "MEMORY", "value": "512"}
        ]
      },
      {
        "name": "xxxxxImage",
        "image": "${var.xxxx_image}",
        "environment": ${jsonencode(concat(
          local.common_env_vars,
          [
            {"name": "apiCall", "value": ""},
            {"name": "wid", "value": ""}
          ]
        ))},
        "essential": true,
        "resourceRequirements": [
          {"type": "VCPU", "value": "0.25"},
          {"type": "MEMORY", "value": "512"}
        ]
      },
      {
        "name": "dataImage",
        "image": "${var.node_server_image}",
        "environment": ${jsonencode(concat(local.common_env_vars, each.value.environment_variables))},
        "essential": true,
        "resourceRequirements": [
          {"type": "VCPU", "value": "0.25"},
          {"type": "MEMORY", "value": "512"}
        ]
      }
  ]
}
CONTAINER_PROPERTIES
}

验证步骤 #

1. 重新部署修改后的状态机和Batch作业定义；
2. 触发Step Functions执行，检查Batch作业的容器日志，确认环境变量已正确传递；
3. 若仍有问题，检查Step Functions执行日志中的Input和Output，确认变量是否正确传入Task节点。

内容的提问来源于stack exchange，提问作者gatorpatch