如何基于变量通过JOLT实现JSON对象的重复生成？

问题背景 #

现有输入JSON、JOLT转换规则及转换后的单条JSON对象，需求是根据输入中amountToCreate字段值（或NiFi的casesToCreate属性值），将转换后的JSON重复对应次数，输出多个相同的JSON对象，询问是否可通过JOLT实现。

现有输入JSON #

{"entityType" : "job","id" : 1019167,"properties" : {"TemplateName" : [ "Development Bug" ],"CaseType" : [ "Internal project" ],"ProjectType" : [ "Development" ],"Team" : [ "" ],"Category" : [ "Bug" ],"WorkflowStepID" : [ "531" ],"ParentCaseID" : 1019167,"CaseName" : [ "Bug development - Test Title" ],"CaseWorkflow" : [ "Development Cycle" ],"Username" : [ "Huvxotup Telfiyi, Ewohuli" ],"CaseDescription" : [ "" ],"CustomerAccount" : "","CustomerInstance" : "","ParentTitle" : "Video test case","Tag" : [ "development.regression.bug" ],"AssignedTo" : [ "" ],"TemplateDescription" : [ "Steps to reproduce: 1. 2. 3. 4. 5. 6.  Current behavior: Expected behavior:  Squealers reports:" ],"Environment" : [ ],"ParentTeam" : "Framework"},"trigger" : "Xguehi Cuwo","amountToCreate" : "3"}

当前使用的JOLT转换规则 #

[{"operation": "shift","spec": {"properties": {"TemplateName|CaseType|ProjectType|Team|Category|WorkflowStepID|CaseName|CaseWorkflow|CaseDescription|Tag|AssignedTo|TemplateDescription|Username": {"*": {"@3,id": "main[&1].ParentCaseID","@": "main[&1].&2","@2,CustomerAccount": "main[&1].CustomerAccount","@2,CustomerInstance": "main[&1].CustomerInstance","@2,ParentTitle": "main[&1].ParentTitle","@2,ParentTeam": "main[&1].ParentTeam"}},"Environment": {"@": "arrays.&"}}}},{"operation": "cardinality","spec": {"main": {"*": {"*": "ONE"}}}},{"operation": "shift","spec": {"main": {"*": {"@2,arrays": {"*": "[&1].&"}},"*": "[&1].&"}}}
]

当前转换输出JSON #

{"Environment" : [ ],"ParentCaseID" : 1019167,"TemplateName" : "Development Bug","CustomerAccount" : "","CustomerInstance" : "","ParentTitle" : "Video test case","ParentTeam" : "Framework","CaseType" : "Internal project","ProjectType" : "Development","Team" : "","Category" : "Bug","WorkflowStepID" : "531","CaseName" : "Bug development - Test Title","CaseWorkflow" : "Development Cycle","CaseDescription" : "","Tag" : "development.regression.bug","AssignedTo" : "","TemplateDescription" : "Steps to reproduce: 1. 2. 3. 4. 5. 6.  Current behavior: Expected behavior:  Squealers reports:","Username" : "Huvxotup Telfiyi, Ewohuli","CasesToCreate" : "2"}

回答 #

JOLT本身不支持根据数值动态生成指定数量的重复JSON对象，因为JOLT是基于静态规则的转换工具，所有转换逻辑都需要提前定义，无法实现动态循环生成N个副本的逻辑。

要实现需求，需要结合NiFi的其他处理器配合完成，推荐两种方案：

方案1：使用ExecuteScript处理器（Groovy脚本） #

1. 先用EvaluateJsonPath处理器提取输入中的amountToCreate值，存储为NiFi文件属性casesToCreate；
2. 用现有JOLT转换规则完成单条JSON对象的结构调整；
3. 添加ExecuteScript处理器，使用Groovy脚本读取casesToCreate属性值，循环对应次数输出转换后的JSON对象副本。示例脚本核心逻辑：

   def count = Integer.parseInt(flowFile.getAttribute("casesToCreate"))
   def json = new String(flowFile.getInputStream().readAllBytes())
   for (int i = 0; i < count; i++) {
       session.write(session.create(flowFile), { out ->
           out.write(json.getBytes())
       } as OutputStreamCallback)
   }
   session.remove(flowFile)
   

方案2：结合SplitJson处理器 #

1. 通过ReplaceText处理器，根据casesToCreate属性值将单条JSON拼接成包含N个重复元素的数组；
2. 用SplitJson处理器拆分该数组，得到多个独立的JSON对象。

两种方案中，ExecuteScript更灵活，适合任意数值的重复需求。

内容的提问来源于stack exchange，提问作者Mariano Diaz