如何通过Jolt实现Person-Property-Feature的反向聚合转换？

需求说明 #

将包含数百个Person对象的JSON数据，转换为以Property为维度的结构：每个Property下按Feature分组，聚合关联的Person ID列表。输入输出示例如下：

输入JSON

[
 {
  "personId": "personId1",
  "properties": [{"id": 1, "name": "property1", "area": "abc"}, {"id": 2, "name": "property2", "area": "def"}],
  "features": [{"featureId": "feature1"}, {"featureId": "feature2"}, {"featureId": "feature3"}]
 },
 {
  "personId": "personId2",
  "properties": [{"id": 1, "name": "property1", "area": "abc"}, {"id": 3, "name": "property3"}],
  "features": [{"featureId": "feature1"}, {"featureId": "feature3"}, {"featureId": "feature5"}]
 }
]

期望输出JSON

[
{
 "id": 1,
 "name": "property1",
 "area": "abc",
 "featureMap": [
 {
 "featureId": "feature1",
 "persons": ["personId1", "personId2"]
 },
 {
 "featureId": "feature2",
 "persons": ["personId1"]
 },
 {
 "featureId": "feature3",
 "persons": ["personId1", "personId2"]
 },
 {
 "featureId": "feature5",
 "persons": ["personId2"]
 }]
},
{
 "id": 2,
 "name": "property2",
 "area": "def",
 "featureMap": [
 {
 "featureId": "feature1",
 "persons": ["personId1"]
 },
 {
 "featureId": "feature2",
 "persons": ["personId1"]
 },
 {
 "featureId": "feature3",
 "persons": ["personId1"]
 }]
},
{
 "id": 3,
 "name": "property3",
 "featureMap": [
 {
 "featureId": "feature1",
 "persons": ["personId2"]
 },
 {
 "featureId": "feature3",
 "persons": ["personId2"]
 },
 {
 "featureId": "feature5",
 "persons": ["personId2"]
 }]
}
]

Jolt转换规则 #

以下是实现需求的多阶段Jolt规范：

[
  // 阶段1：展开所有Property-Feature-Person的关联条目
  {
    "operation": "shift",
    "spec": {
      "*": {
        "properties": {
          "*": {
            "@2,personId": "[&1].persons[]",
            "@2,features": "[&1].features",
            "id": "[&1].property.id",
            "name": "[&1].property.name",
            "area": "[&1].property.area"
          }
        }
      }
    }
  },
  // 阶段2：将每个Property的Feature展开，关联对应的PersonId
  {
    "operation": "shift",
    "spec": {
      "*": {
        "features": {
          "*": {
            "featureId": "[&3].property.features[&1].featureId",
            "@2,persons": "[&3].property.features[&1].persons"
          }
        },
        "property": {
          "*": "[&2].property.&"
        }
      }
    }
  },
  // 阶段3：按Property的id分组，聚合Feature对应的Person列表
  {
    "operation": "shift",
    "spec": {
      "*": {
        "property": {
          "features": {
            "*": {
              "featureId": "groupedByPropertyId[@(3,id)].featureMap[&1].featureId",
              "persons": "groupedByPropertyId[@(3,id)].featureMap[&1].persons"
            }
          },
          "*": "groupedByPropertyId[@(1,id)].&"
        }
      }
    }
  },
  // 阶段4：将分组后的结构转为数组，完成最终格式
  {
    "operation": "shift",
    "spec": {
      "groupedByPropertyId": {
        "*": "[]"
      }
    }
  }
]

转换步骤解释 #

1. 阶段1：遍历每个Person的所有Property，将PersonId、Property的元数据（id/name/area）以及该Person的所有Feature关联起来，生成中间条目。
2. 阶段2：将每个中间条目中的Feature数组展开，让每个Feature单独关联对应的PersonId，为后续聚合做准备。
3. 阶段3：以Property的id为键进行分组，将同一个Feature下的PersonId收集为数组，同时保留Property的其他元数据。
4. 阶段4：将分组后的对象结构转换为数组，得到最终的输出格式。

该规则支持Property的自动去重（通过id分组），同时完成Feature与Person列表的聚合，适用于数百条Person数据的场景。

内容的提问来源于stack exchange，提问作者Pankaj Agarwal