能否通过Jolt转换实现记录过滤、字段移除与结构重组？

完全可以实现你需要的JSON转换，以下是对应的Jolt规格：

[
  {
    "operation": "shift",
    "spec": {
      "records": {
        "*": {
          // 标记需剔除的字段
          "idNumber": null,
          "accountNumber": null,
          // 映射顶层公共字段到每条记录
          "@(2,messageId)": "[&1].messageId",
          "@(2,markerId)": "[&1].markerId",
          "@(2,dateFrom)": "[&1].dateFrom",
          "@(2,dateTo)": "[&1].dateTo",
          // 保留基础字段
          "recordId": "[&1].recordId",
          "account": "[&1].account",
          "city": "[&1].city",
          // 展开data嵌套字段到当前层级
          "data": {
            "*": "[&2].&"
          }
        }
      }
    }
  },
  {
    "operation": "remove",
    "spec": {
      "*": {
        "idNumber": "",
        "accountNumber": ""
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": "=notNull"
    }
  }
]

转换逻辑说明： #

1. Shift阶段：遍历records数组，将顶层公共字段映射到每条记录，保留目标字段并展开data嵌套内容，同时标记含idNumber/accountNumber的字段为待清理项。
2. Remove阶段：彻底移除所有记录中存在的idNumber和accountNumber字段。
3. Modify阶段：过滤掉转换后为空的对象，确保最终输出仅保留有效记录。

输入JSON： #

{
  "messageId": 1234,
  "markerId": "T",
  "dateFrom": 6436058131202690000,
  "dateTo": -3840351829778683400,
  "records": [
    {
      "recordId": 1,
      "account": "152739203233"
    },
    {
      "recordId": 2,
      "data": {
        "email": "jsmith@gmail.com",
        "firstName": "John",
        "lastName": "Smith"
      }
    },
    {
      "recordId": 3,
      "city": "Los Angeles"
    },
    {
      "recordId": 4,
      "idNumber": "12345"
    },
    {
      "recordId": 5,
      "accountNumber": "55671"
    },
    {
      "recordId": 6,
      "account": "6789189790191"
    },
    {
      "recordId": 7,
      "city": "San Fransisco"
    }
  ]
}

输出结果： #

[
  {
    "messageId": 1234,
    "markerId": "T",
    "dateFrom": 6436058131202690000,
    "dateTo": -3840351829778683400,
    "recordId": 1,
    "account": "152739203233"
  },
  {
    "messageId": 1234,
    "markerId": "T",
    "dateFrom": 6436058131202690000,
    "dateTo": -3840351829778683400,
    "recordId": 2,
    "email": "jsmith@gmail.com",
    "firstName": "John",
    "lastName": "Smith"
  },
  {
    "messageId": 1234,
    "markerId": "T",
    "dateFrom": 6436058131202690000,
    "dateTo": -3840351829778683400,
    "recordId": 3,
    "city": "Los Angeles"
  },
  {
    "messageId": 1234,
    "markerId": "T",
    "dateFrom": 6436058131202690000,
    "dateTo": -3840351829778683400,
    "recordId": 6,
    "account": "6789189790191"
  },
  {
    "messageId": 1234,
    "markerId": "T",
    "dateFrom": 6436058131202690000,
    "dateTo": -3840351829778683400,
    "recordId": 7,
    "city": "San Fransisco"
  }
]

内容的提问来源于stack exchange，提问作者Sophie78