Jolt转换无输出问题：按指定字段合并JSON数据需求

需求 #

编写Jolt规则转换JSON数据，当前规则无输出且未达预期，具体需求：

• 多条数据的practice_loc、prac_num、topId字段完全相同时，将它们的S1、S2字段存入subList数组的子对象中，字段名保留S1/S2；
• 单条数据时，为其添加subList数组，数组内子对象需将S1/S2映射为SubID1/SubID2。

原始数据 #

[
  {
    "practice_loc": "120",
    "prac_num": "oswal",
    "topId": "t1",
    "S1": "A1",
    "S2": "B1"
  },
  {
    "practice_loc": "120",
    "prac_num": "oswal",
    "topId": "t1",
    "S1": "A2",
    "S2": ""
  },
  {
    "practice_loc": "334",
    "prac_num": "L3",
    "topId": "plumcherry",
    "S1": "A3",
    "S2": ""
  },
  {
    "practice_loc": "987",
    "prac_num": "L3",
    "topId": "artica",
    "S1": "A5",
    "S2": "B7"
  }
]

预期输出 #

[
  {
    "practice_loc": "120",
    "prac_num": "oswal",
    "topId": "t1",
    "subList": [
      {
        "S1": "A1",
        "S2": "B1"
      },
      {
        "S1": "A2",
        "S2": ""
      }
    ]
  },
  {
    "practice_loc": "334",
    "prac_num": "L3",
    "topId": "plumcherry",
    "subList": [
      {
        "SubID1": "A3",
        "SubID2": ""
      }
    ]
  },
  {
    "practice_loc": "987",
    "prac_num": "L3",
    "topId": "artica",
    "subList": [
      {
        "SubID1": "A5",
        "SubID2": "B7"
      }
    ]
  }
]

尝试过的无效规则（无输出） #

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "@": "@(1,practice_loc).@(1,prac_num).@(1,topId)"
      }
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "*": {
        "*": "MANY"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "*": {
            "practice_loc": "[#4].&",
            "prac_num": "[#4].&",
            "topId": "[#4].&",
            "S*": "[#4].subList[&1].&"
          }
        }
      }
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "*": {
        "practice_loc": "ONE",
        "prac_num": "ONE",
        "topId": "ONE"
      }
    }
  }
]

修正后的Jolt规则 #

[
  // 1. 按practice_loc、prac_num、topId拼接唯一键分组，聚合同组数据
  {
    "operation": "shift",
    "spec": {
      "*": {
        "@": "@(1,practice_loc)_@(1,prac_num)_@(1,topId).items[]",
        "practice_loc": "@(1,practice_loc)_@(1,prac_num)_@(1,topId).practice_loc",
        "prac_num": "@(1,practice_loc)_@(1,prac_num)_@(1,topId).prac_num",
        "topId": "@(1,practice_loc)_@(1,prac_num)_@(1,topId).topId"
      }
    }
  },
  // 2. 生成subList，根据items数组长度判断字段映射规则
  {
    "operation": "shift",
    "spec": {
      "*": {
        "practice_loc": "[#2].&",
        "prac_num": "[#2].&",
        "topId": "[#2].&",
        "items": {
          "*": {
            "S1": "[#4].subList[&1].@(3,items.length>1?S1:SubID1)",
            "S2": "[#4].subList[&1].@(3,items.length>1?S2:SubID2)"
          }
        }
      }
    }
  },
  // 3. 确保外层practice_loc、prac_num、topId为单值
  {
    "operation": "cardinality",
    "spec": {
      "*": {
        "practice_loc": "ONE",
        "prac_num": "ONE",
        "topId": "ONE"
      }
    }
  }
]

规则说明 #

1. 分组聚合：用三个字段拼接成唯一标识键，将同组数据存入items数组，同时单独存储三个分组字段，避免后续重复提取。
2. 字段映射与subList生成：遍历每个分组，提取外层字段后，根据items数组长度判断：长度大于1时保留原S1/S2字段名，否则映射为SubID1/SubID2，并将对应值存入subList数组。
3. 去重处理：通过cardinality操作确保外层的三个分组字段仅保留一个有效值。

内容的提问来源于stack exchange，提问作者dracile