如何在Apache NiFi中编写Jolt规范实现JSON嵌套转换

我正在使用Apache NiFi的JoltTransformation处理器，将扁平JSON数据转换为指定的嵌套JSON格式。需求是按Agent分组，同时构建Company、Model1、Sub-Model1的层级结构，但之前写的Jolt规范达不到预期，寻求正确的编写方法。

输入JSON #

[
  {
    "Agent": "A1",
    "Location": "L1",
    "Company": "Hyundai",
    "Model1": "Verna",
    "Sub-Model1": "2018"
  },
  {
    "Agent": "A1",
    "Location": "L1",
    "Company": "Hyundai",
    "Model1": "Creta",
    "Sub-Model1": "2015"
  },
  {
    "Agent": "A1",
    "Location": "L1",
    "Company": "Hyundai",
    "Model1": "Aura",
    "Sub-Model1": "2022"
  },
  {
    "Agent": "A2",
    "Location": "L1",
    "Company": "Toyota",
    "Model1": "Fortuner",
    "Sub-Model1": "2020"
  }
]

期望嵌套输出 #

[
  {
    "Agent": "A1",
    "loc_id": "L1",
    "Company": {
      "Hyundai": [
        {
          "Verna": [
            {
              "2018": []
            }
          ]
        },
        {
          "Creta": [
            {
              "2015": []
            }
          ]
        },
        {
          "Aura": [
            {
              "2022": []
            }
          ]
        }
      ]
    }
  },
  {
    "Agent": "A2",
    "loc_id": "L1",
    "Company": {
      "Toyota": [
        {
          "Fortuner": [
            {
              "2020": []
            }
          ]
        }
      ]
    }
  }
]

我尝试过的Jolt规范（无效） #

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": "@(1,Agent)@(1,location)@(1,Company)@(1,Model1)@(1,Sub-Model1).&"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": {
        "$": "agent.[#2].Agent",
        "*": {
          "$": "agent.[#3].location",
          "*": {
            "*": {
              "$": "agent.[#4].Company",
              "*": {
                "Model1*": "agent.[#6].Company.[#4].Sub-MOdel1.[#2].&"
              }
            }
          }
        }
      }
    }
  }
]

正确的Jolt规范 #

下面的Jolt规范可以实现预期的嵌套结构，通过三次操作完成分组、层级梳理和格式补全：

[
  // 第一步：按Agent分组，整理基础字段并搭建层级雏形
  {
    "operation": "shift",
    "spec": {
      "*": {
        "Agent": "@(1,Agent).Agent",
        "Location": "@(1,Agent).loc_id",
        "Company": "@(1,Agent).Company.@(1,Company).[#2].@(1,Model1).[0].@(1,Sub-Model1)"
      }
    }
  },
  // 第二步：将分组后的对象结构转为数组格式，明确层级路径
  {
    "operation": "shift",
    "spec": {
      "*": {
        "Agent": "[#2].Agent",
        "loc_id": "[#2].loc_id",
        "Company": {
          "*": {
            "*": {
              "*": {
                "*": "[#6].Company.&4.[#4].&2.[0].&"
              }
            }
          }
        }
      }
    }
  },
  // 第三步：将最内层年份字段设置为空数组
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "Company": {
          "*": {
            "*": {
              "*": {
                "*": []
              }
            }
          }
        }
      }
    }
  }
]

操作说明 #

1. 第一次shift：以Agent为核心键分组，将Location重命名为loc_id，同时把Company、Model1、Sub-Model1的层级关系初步映射。
2. 第二次shift：把按Agent分组的对象结构转换为数组，确保每个Agent对应数组中的一个元素，同时梳理好各层级的路径对应关系。
3. modify-overwrite-beta：将最内层年份字段的原始值替换为空数组，完全匹配期望输出的格式要求。

内容的提问来源于stack exchange，提问作者Destiel