基于stopType聚合的Flat JSON转Nested JSON的Jolt转换需求

需要将包含多条stop记录的扁平JSON，按stopType字段聚合为单个嵌套对象——顶层保留各条记录的公共字段，不同stopType的专属字段作为对应名称的子对象。

输入JSON #

[
  {
    "container_id": "ABC",
    "shipperN": null,
    "PNumber": null,
    "trackingNumber": null,
    "priority": null,
    "loadNumber": "123345",
    "billOfLading": "12345",
    "addressLine1": "ABC Street",
    "addressLine2": null,
    "city": "Chicago",
    "country": "US",
    "latitude": null,
    "longitude": null,
    "earliestAppointmentTime": "XXXXX09:25",
    "latestAppointmentTime": "XXXXX09:25",
    "postalCode": "XXXXX3",
    "sequence": "1",
    "state": "XY",
    "stopReferenceId": "0001",
    "stopType": "PU",
    "containerNumber": "XXXXXXXX"
  },
  {
    "container_id": "ABC",
    "shipperN": null,
    "PNumber": null,
    "trackingNumber": null,
    "priority": null,
    "loadNumber": "123345",
    "billOfLading": "12345",
    "addressLine1": null,
    "addressLine2": null,
    "city": null,
    "country": null,
    "latitude": null,
    "longitude": null,
    "earliestAppointmentTime": "XXXXX09:25",
    "latestAppointmentTime": "XXXXX09:25",
    "name": null,
    "postalCode": null,
    "sequence": "2",
    "state": null,
    "stopReferenceId": "XXXXD",
    "stopType": "PL",
    "containerNumber": "XXXXXXXX"
  }
]

期望输出JSON #

[
  {
    "container_id": "ABC",
    "shipperN": null,
    "PNumber": null,
    "trackingNumber": null,
    "priority": null,
    "loadNumber": "123345",
    "billOfLading": "12345",
    "PU": {
      "addressLine1": "ABC Street",
      "city": "Chicago",
      "country": "US",
      "earliestAppointmentTime": "XXXXX09:25",
      "latestAppointmentTime": "XXXXX09:25",
      "postalCode": "XXXXX3",
      "sequence": "1",
      "state": "XY",
      "stopReferenceId": "0001",
      "stopType": "PU"
    },
    "PL": {
      "earliestAppointmentTime": "XXXXX09:25",
      "latestAppointmentTime": "XXXXX09:25",
      "sequence": "2",
      "stopReferenceId": "dummy",
      "stopType": "PL",
      "externalAddressId": "dummy"
    },
    "containerNumber": "XXXXXXXX"
  }
]

Jolt转换规则 #

[
  // 第一阶段：拆分公共字段与stopType子对象
  {
    "operation": "shift",
    "spec": {
      "*": {
        "container_id": "[0].container_id",
        "shipperN": "[0].shipperN",
        "PNumber": "[0].PNumber",
        "trackingNumber": "[0].trackingNumber",
        "priority": "[0].priority",
        "loadNumber": "[0].loadNumber",
        "billOfLading": "[0].billOfLading",
        "containerNumber": "[0].containerNumber",
        "stopType": "[0].&.&stopType",
        "addressLine1": "[0].&1.addressLine1",
        "city": "[0].&1.city",
        "country": "[0].&1.country",
        "earliestAppointmentTime": "[0].&1.earliestAppointmentTime",
        "latestAppointmentTime": "[0].&1.latestAppointmentTime",
        "postalCode": "[0].&1.postalCode",
        "sequence": "[0].&1.sequence",
        "state": "[0].&1.state",
        "stopReferenceId": "[0].&1.stopReferenceId"
      }
    }
  },
  // 第二阶段：处理PL的特殊字段替换与新增
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "PL": {
          "stopReferenceId": "dummy",
          "externalAddressId": "dummy"
        }
      }
    }
  },
  // 第三阶段：移除子对象中的null值（按需保留可删除此阶段）
  {
    "operation": "remove",
    "spec": {
      "*": {
        "*": {
          "*": [""]
        }
      }
    }
  }
]

规则说明 #

1. Shift阶段：提取所有公共字段到顶层，通过&1引用当前stopType值，将每条记录的stop相关字段映射到对应名称的子对象中；
2. Modify阶段：针对PL子对象，替换stopReferenceId为dummy并新增externalAddressId字段；
3. Remove阶段：清除子对象中值为null的字段，让输出更简洁。

内容的提问来源于stack exchange，提问作者VamsiKumar_gudala