JOLT转换：处理含null的多JSON数组合并为单数组问题

问题说明 #

此前已在Stack Overflow提问《JOLT Transform : Extract and combine fields from multiple JSON arrays to single array output》，当前JOLT规范在两个数组均非空时可正常输出，但当任一数组为null时会输出null，不符合预期。需调整规范，确保其中一个数组为null时，仍能基于非空数组的长度生成对应输出，空数组对应的字段设为null。

示例1：responses为null，answers非空 #

输入 #

{
  "questionResponses": [
    {
      "responses": null
    }
  ],
  "answers": [
    {
      "questionId": "1",
      "answerId": "answer 1"
    },
    {
      "questionId": "1",
      "answerId": "answer 2"
    }
  ],
  "totalAttachments": 0
}

非期望输出 #

null

预期输出 #

[
  {
    "response": null,
    "answerId": "answer 1"
  },
  {
    "response": null,
    "answerId": "answer 2"
  }
]

示例2：answers为null，responses非空 #

输入 #

{
  "questionResponses": [
    {
      "responses": [
        {
          "responseId": "1",
          "response": "response 1"
        },
        {
          "responseId": "2",
          "response": "response 2"
        },
        {
          "responseId": "3",
          "response": "response 3"
        }
      ]
    }
  ],
  "answers": null,
  "totalAttachments": 0
}

非期望输出 #

null

预期输出 #

[
  {
    "response": "response 1",
    "answerId": null
  },
  {
    "response": "response 2",
    "answerId": null
  },
  {
    "response": "response 3",
    "answerId": null
  }
]

当前使用的JOLT规范 #

[
  {
    "operation": "shift",
    "spec": {
      "questionR*": {
        "*": {
          "responses": {
            "*": {
              "@(4,answers)": "@(1,response)"
            }
          }
        }
      }
    }
  },
  {
    // Determine two independent arrays both with size of 6
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "$1": "response",
          "a*": "&"
        }
      }
    }
  },
  {
    // expand those arrays as objects which contain desired key-value pairs
    "operation": "shift",
    "spec": {
      "*": {
        "*": "&.&1"
      }
    }
  },
  {
    // get rid of object labels
    "operation": "shift",
    "spec": {
      "*": ""
    }
  }
]

修正后的JOLT规范 #

[
  {
    // 提取responses和answers的核心数据，同时初始化空数组处理null场景
    "operation": "shift",
    "spec": {
      "questionResponses": {
        "*": {
          "responses": {
            "*": {
              "response": "res[&1].response"
            }
          }
        }
      },
      "answers": {
        "*": {
          "answerId": "ans[&1].answerId"
        }
      }
    }
  },
  {
    // 处理null数组：若其中一个数组为空，生成对应长度的空值元素
    "operation": "modify-overwrite-beta",
    "spec": {
      "res": ["=notNull", []],
      "ans": ["=notNull", []],
      "resSize": "=size(@(1,res))",
      "ansSize": "=size(@(1,ans))",
      "tempRes": "=if(@(1,resSize)==0, repeat([{\"response\":null}],@(1,ansSize)), @(1,res))",
      "tempAns": "=if(@(1,ansSize)==0, repeat([{\"answerId\":null}],@(1,resSize)), @(1,ans))"
    }
  },
  {
    // 执行笛卡尔积（兼容原逻辑：两个数组非空时生成全组合）
    "operation": "shift",
    "spec": {
      "tempRes": {
        "*": {
          "@": "@(1,response).&",
          "@(2,tempAns)": "@(1,response).&1.ans"
        }
      }
    }
  },
  {
    // 展开笛卡尔积结构，合并response和answerId字段
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "response": "&2.response",
          "ans": {
            "*": {
              "answerId": "&3.answerId"
            }
          }
        }
      }
    }
  },
  {
    // 整理为最终数组格式
    "operation": "shift",
    "spec": {
      "*": ""
    }
  }
]

修正逻辑说明 #

1. 提取与初始化：先提取responses和answers的核心字段，并用=notNull将null转换为空数组，避免后续操作因null中断。
2. 空数组处理：通过modify-overwrite-beta判断数组长度，若其中一个数组为空，生成与另一个数组长度一致的、对应字段为null的元素数组。
3. 笛卡尔积兼容：保留原逻辑中两个数组非空时的笛卡尔积组合，确保原有场景不受影响。
4. 结构整理：展开嵌套结构，将最终结果整理为预期的数组格式。

内容的提问来源于stack exchange，提问作者Vectors