Jolt转换需求：缺失字段设为null并保留原位置

现有Jolt转换规则在输入字段完整时可正常转换，但当存在缺失字段（如示例中的Gender）时，需要将缺失字段设为null并保留在指定的输出位置（此前设置默认null会被放到字段末尾）。

现有Jolt转换规格 #

[
  {
    "operation": "shift",
    "spec": {
      "data": {
        "*": {
          "id": "[&1].id",
          "Test_code": "[&1].Test_code",
          "applyData": {
            "Email": "[&2].Email",
            "Address": "[&2].Address",
            "Gender": "[&2].Gender",
            "Date": "[&2].Date",
            "Industry": "[&2].Industry",
            "StartDate": "[&2].StartDate",
            "TotalExperience": "[&2].TotalExperience"
          }
        }
      }
    }
  }
]

完整字段的输入示例 #

{
  "data": [
    {
      "id": "124354",
      "applicantId": "10775119",
      "Test_code": "123454",
      "applyData": {
        "Email": "blabla@gmail.com",
        "Address": null,
        "Gender": "M",
        "Date": "2024-01-28 23:23:39",
        "Industry": "IT-Software",
        "StartDate": "2013-04-01",
        "TotalExperience": "11"
      }
    }
  ]
}

缺失字段的输入示例 #

{
  "data": [
    {
      "id": "124354",
      "applicantId": "10775119",
      "Test_code": "123454",
      "applyData": {
        "Email": "blabla@gmail.com",
        "Address": null,
        "Date": "2024-01-28 23:23:39",
        "Industry": "IT-Software",
        "StartDate": "2013-04-01",
        "TotalExperience": "11"
      }
    }
  ]
}

期望输出 #

[
  {
    "id": "124354",
    "Test_code": "123454",
    "Email": "blabla@gmail.com",
    "PermanentAddress": null,
    "Gender": null,
    "Date": "2024-01-28 23:23:39",
    "Industry": "IT-Software",
    "StartDate": "2013-04-01",
    "TotalExperience": "11"
  }
]

解决方案 #

要实现缺失字段保留指定位置并设为null，需要结合shift和default操作：先通过shift映射字段并指定输出顺序，再用default为所有必填字段设置默认null值。

修改后的Jolt规格如下：

[
  // 第一步：映射字段并指定输出顺序，将原Address映射为期望的PermanentAddress
  {
    "operation": "shift",
    "spec": {
      "data": {
        "*": {
          "id": "[&1].id",
          "Test_code": "[&1].Test_code",
          "applyData": {
            "Email": "[&2].Email",
            "Address": "[&2].PermanentAddress",
            "Gender": "[&2].Gender",
            "Date": "[&2].Date",
            "Industry": "[&2].Industry",
            "StartDate": "[&2].StartDate",
            "TotalExperience": "[&2].TotalExperience"
          }
        }
      }
    }
  },
  // 第二步：为所有需要保留的字段设置默认null
  {
    "operation": "default",
    "spec": {
      "*": {
        "id": null,
        "Test_code": null,
        "Email": null,
        "PermanentAddress": null,
        "Gender": null,
        "Date": null,
        "Industry": null,
        "StartDate": null,
        "TotalExperience": null
      }
    }
  }
]

关键说明 #

1. shift操作中显式定义了输出字段的顺序，Jolt会严格按照该顺序生成输出对象的键。
2. default操作针对数组中的每个元素（*），为所有目标字段设置默认null，确保缺失字段也会在指定位置生成。
3. 已将原输入中的Address映射为期望输出的PermanentAddress，匹配需求中的字段名称变更。

内容的提问来源于stack exchange，提问作者Sinuers