Jolt转换输出数组存在null值问题及修正需求

输入JSON #

[
  {
    "referencetype": "ProductToProductPlatformReferenceLink",
    "objecttype": "Product Platform",
    "id": "PP_W",
    "name": "Wiley",
    "values": {
      "EventTrigger": "",
      "PlatformID": "W",
      "PlatformName": "Wiley",
      "PlatformURL": "www.wiley.com"
    },
    "referenceValues": {},
    "parent": []
  },
  {
    "referencetype": "ProductToSubjectHierarchyLink",
    "objecttype": "Subject Code",
    "id": "SC_LS01",
    "name": "General Biology",
    "values": {
      "EventTrigger": "",
      "Level2Name": [
        "General Life Sciences"
      ],
      "OLCodeName": [
        "General Biology"
      ],
      "SubjectCode": "LS01",
      "SubjectCodeName": [
        "General Biology"
      ],
      "SubjectGroup": [
        "0009"
      ],
      "SubjectGroupName": [
        "Life Sciences"
      ],
      "SubjectLevel2": [
        "000061"
      ],
      "SubjectOnlineCode": [
        "LS01"
      ]
    },
    "referenceValues": {
      "SubjectCodeSequence": ""
    },
    "parent": []
  },
  {
    "referencetype": "ProductToSubjectHierarchyLink",
    "objecttype": "Subject Code",
    "id": "SC_LS30",
    "name": "Cell & Molecular Biology",
    "values": {
      "EventTrigger": "",
      "Level2Name": [
        "Cell & Molecular Biology"
      ],
      "OLCodeName": [
        "Cell & Molecular Biology"
      ],
      "SubjectCode": "LS30",
      "SubjectCodeName": [
        "Cell & Molecular Biology"
      ],
      "SubjectGroup": [
        "0009"
      ],
      "SubjectGroupName": [
        "Life Sciences"
      ],
      "SubjectLevel2": [
        "000030"
      ],
      "SubjectOnlineCode": [
        "LS30"
      ]
    },
    "referenceValues": {
      "SubjectCodeSequence": ""
    },
    "parent": []
  },
  {
    "referencetype": "ProductToSubjectHierarchyLink",
    "objecttype": "Subject Code",
    "id": "SC_LS36",
    "name": "Biochemistry",
    "values": {
      "EventTrigger": "",
      "Level2Name": [
        "Cell & Molecular Biology"
      ],
      "OLCodeName": [
        "Biochemistry"
      ],
      "SubjectCode": "LS36",
      "SubjectCodeName": [
        "Biochemistry"
      ],
      "SubjectGroup": [
        "0009"
      ],
      "SubjectGroupName": [
        "Life Sciences"
      ],
      "SubjectLevel2": [
        "000030"
      ],
      "SubjectOnlineCode": [
        "LS36"
      ]
    },
    "referenceValues": {
      "SubjectCodeSequence": ""
    },
    "parent": []
  }
]

预期输出 #

{
  "subject": [
    {
      "subjectCategoryLevel2": "000061",
      "subjectCategoryLevel2Name": "General Life Sciences",
      "subjectCategoryWOLCode": "LS01",
      "subjectCategoryWOLName": "General Biology",
      "subjectCategory": "LS01",
      "subjectCategoryName": "General Biology",
      "subjectCodeSequence": "",
      "subjectCategoryGroup": "0009",
      "subjectCategoryGroupName": "Life Sciences"
    },
    {
      "subjectCategoryLevel2": "000030",
      "subjectCategoryLevel2Name": "Cell & Molecular Biology",
      "subjectCategoryWOLCode": "LS30",
      "subjectCategoryWOLName": "Cell & Molecular Biology",
      "subjectCategory": "LS30",
      "subjectCategoryName": "Cell & Molecular Biology",
      "subjectCodeSequence": "",
      "subjectCategoryGroup": "0009",
      "subjectCategoryGroupName": "Life Sciences"
    },
    {
      "subjectCategoryLevel2": "000030",
      "subjectCategoryLevel2Name": "Cell & Molecular Biology",
      "subjectCategoryWOLCode": "LS36",
      "subjectCategoryWOLName": "Biochemistry",
      "subjectCategory": "LS36",
      "subjectCategoryName": "Biochemistry",
      "subjectCodeSequence": "",
      "subjectCategoryGroup": "0009",
      "subjectCategoryGroupName": "Life Sciences"
    }
  ]
}

当前使用的Jolt Spec #

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "referencetype": {
          "ProductToSubjectHierarchyLink": {
            "@2": {
              "values": {
                "SubjectLevel2": {
                  "0": "subject[&6].subjectCategoryLevel2"
                },
                "Level2Name": {
                  "0": "subject[&6].subjectCategoryLevel2Name"
                },
                "OLCodeName": {
                  "0": "subject[&6].subjectCategoryWOLName"
                },
                "SubjectCodeName": {
                  "0": "subject[&6].subjectCategoryName"
                },
                "SubjectCode": "subject[&5].subjectCategoryWOLCode",
                "SubjectGroup": {
                  "0": "subject[&6].subjectCategoryGroup"
                },
                "SubjectGroupName": {
                  "0": "subject[&6].subjectCategoryGroupName"
                },
                "SubjectOnlineCode": {
                  "0": "subject[&6].subjectCategory"
                }
              },
              "referenceValues": {
                "SubjectCodeSequence": "subject[&5].subjectCodeSequence"
              }
            }
          }
        }
      }
    }
  }
]

转换后错误结果 #

{
  "subject": [
    null,
    {
      "subjectCategoryLevel2": "000061",
      "subjectCategoryLevel2Name": "General Life Sciences",
      "subjectCategoryWOLName": "General Biology",
      "subjectCategoryName": "General Biology",
      "subjectCategoryWOLCode": "LS01",
      "subjectCategoryGroup": "0009",
      "subjectCategoryGroupName": "Life Sciences",
      "subjectCategory": "LS01",
      "subjectCodeSequence": ""
    },
    {
      "subjectCategoryLevel2": "000030",
      "subjectCategoryLevel2Name": "Cell & Molecular Biology",
      "subjectCategoryWOLName": "Cell & Molecular Biology",
      "subjectCategoryName": "Cell & Molecular Biology",
      "subjectCategoryWOLCode": "LS30",
      "subjectCategoryGroup": "0009",
      "subjectCategoryGroupName": "Life Sciences",
      "subjectCategory": "LS30",
      "subjectCodeSequence": ""
    }
  ]
}

问题原因 #

1. null值产生：原Spec中使用subject[&6]绑定了原数组的索引位置，第一个元素属于ProductToProductPlatformReferenceLink类型，不符合过滤条件，导致该索引位置被填充为null。
2. 元素缺失：层级引用&6、&5的计算错误，导致第三个符合条件的SC_LS36对象未被正确映射到结果数组中。

修正后的Jolt Spec #

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "referencetype": {
          "ProductToSubjectHierarchyLink": {
            "@2": {
              "values": {
                "SubjectLevel2[0]": "subject[].subjectCategoryLevel2",
                "Level2Name[0]": "subject[].subjectCategoryLevel2Name",
                "OLCodeName[0]": "subject[].subjectCategoryWOLName",
                "SubjectCodeName[0]": "subject[].subjectCategoryName",
                "SubjectCode": "subject[].subjectCategoryWOLCode",
                "SubjectGroup[0]": "subject[].subjectCategoryGroup",
                "SubjectGroupName[0]": "subject[].subjectCategoryGroupName",
                "SubjectOnlineCode[0]": "subject[].subjectCategory"
              },
              "referenceValues": {
                "SubjectCodeSequence": "subject[].subjectCodeSequence"
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "subject": {
        "*": {
          "*": "ONE"
        }
      }
    }
  }
]

修正说明 #

1. 消除null值：将subject[&6]改为subject[]，让Jolt自动为符合条件的元素分配数组位置，不会为不符合条件的元素预留位置。
2. 简化数组取值：直接用xxx[0]的方式提取数组第一个元素，避免层级引用错误。
3. 添加cardinality操作：确保每个字段只保留一个值，避免因多路径赋值导致的数组重复问题。

执行修正后的Spec后，输出结果将完全符合预期，无null值且包含所有3个Subject Code对象。

内容的提问来源于stack exchange，提问作者Sayan Mukherjee