如何使用Jolt扁平化包含新旧订单的嵌套JSON？

问题背景 #

现有包含新旧订单信息的嵌套JSON数据，后续可能新增非嵌套字段，但整体层级结构保持不变。需要将其扁平化为完整JSON路径作为键的结构，硬编码Jolt规则无法适配新增字段，需通用解决方案。

输入JSON示例 #

{
  "data": {
    "new": {
      "orders": {
        "type": "atype",
        "currency": "usd",
        "name": null,
        "code": null,
        "shipped": null,
        "identifier": "some uuid",
        "source": "webshop",
        "productinfo": [
          {
            "pType": "fan",
            "ts": "2024-11-15T13:33:16.047Z",
            "status": null,
            "seller": {
              "identifier": "uuid",
              "internacodes": [
                {
                  "type": "US",
                  "value": "1111"
                },
                {
                  "type": "UUID",
                  "value": "some uuid"
                }
              ]
            }
          },
          {
            "pType": "fan",
            "ts": "2024-11-25T13:33:16.047Z",
            "status": null,
            "seller": {
              "identifier": "uuid",
              "internacodes": [
                {
                  "type": "US",
                  "value": "1111"
                },
                {
                  "type": "UUID",
                  "value": "some uuid"
                }
              ]
            }
          }
        ],
        "unique_num": [
          {
            "type": "UUID",
            "value": "some uuid",
            "ts": "2024-11-13T25:15:47.919Z"
          },
          {
            "type": "product id",
            "value": "23456",
            "ts": "2024-11-22T15:15:46.912Z"
          }
        ]
      }
    },
    "old": {
      "orders": {
        "type": "atype",
        "currency": "usd",
        "name": null,
        "code": null,
        "shippedOffice": null,
        "identifier": "some uuid",
        "source": "webshop",
        "productinfo": [
          {
            "pType": "fan",
            "ts": "2024-11-15T13:33:16.047Z",
            "status": null,
            "seller": {
              "identifier": "uuid",
              "internacodes": [
                {
                  "type": "US",
                  "value": "1111"
                },
                {
                  "type": "UUID",
                  "value": "some uuid"
                }
              ]
            }
          }
        ],
        "unique_num": [
          {
            "type": "UUID",
            "value": "some uuid",
            "ts": "2024-11-13T25:15:47.919Z"
          },
          {
            "type": "product id",
            "value": "23456",
            "ts": "2024-11-22T15:15:46.912Z"
          }
        ]
      }
    }
  }
}

期望输出示例 #

{
  "data.new.orders.type": "atype",
  "data.new.orders.currency": "usd",
  "data.new.orders.name": null,
  "data.new.orders.productinfo[0].pType": "fan",
  "data.new.orders.productinfo[0].seller.identifier": "uuid",
  "data.old.orders.shippedOffice": null,
  ...
}

通用Jolt解决方案 #

使用如下Jolt规范，无需硬编码字段名，自动适配层级内新增的非嵌套字段：

[
  {
    "operation": "shift",
    "spec": {
      "data": {
        "*": { // 匹配new/old及未来新增的同级字段
          "orders": {
            "*": { // 匹配orders下所有字段（含新增）
              // 处理非数组字段
              "@(3,data).@(2,&1).@(1,&)": "@(3,data).@(2,&1).@(1,&)",
              // 处理数组类型字段，递归展开
              "*": {
                "*": {
                  "@(5,data).@(4,&2).@(3,&1).@(2,&).@(1,&)": "@(5,data).@(4,&2).@(3,&1).@(2,&).@(1,&)",
                  "*": {
                    "@(6,data).@(5,&2).@(4,&1).@(3,&).@(2,&).@(1,&)": "@(6,data).@(5,&2).@(4,&1).@(3,&).@(2,&).@(1,&)"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
]

方案说明 #

1. 动态匹配字段：用*通配符匹配data下的所有子节点（如new/old），以及orders下的所有字段，新增字段会被自动捕获
2. 路径拼接：通过@(n,&)语法向上引用层级键名，拼接成完整的JSON路径作为输出键（@(3,data)表示向上3层取data键值，@(2,&1)表示向上2层取当前节点的键名）
3. 数组处理：针对嵌套数组（如productinfo、internacodes），通过多层嵌套的*匹配数组元素，路径中会自动包含数组索引（如productinfo[0]）
4. 层级兼容：只要原JSON的核心层级（data -> [xxx] -> orders）不变，新增的非嵌套字段无需修改Jolt规则即可被扁平化

内容的提问来源于stack exchange，提问作者Dot Net Dev 19