如何在NiFi中将动态字段的JSON对象转换为键值对数组

Hey there! Converting a dynamic JSON object into a key-value array is a straightforward task with NiFi's built-in processors—no custom scripts required. Here's the step-by-step solution using JoltTransformJSON, which handles dynamic fields perfectly:

The core tool here is the JoltTransformJSON processor, which excels at flexible JSON transformations, including dynamic field mapping.

Step 1: Add and Configure JoltTransformJSON #

1. Drag the JoltTransformJSON processor onto your NiFi canvas and connect it to your JSON data source.
2. Open the processor's configuration panel and navigate to the Jolt Specification property.
3. Paste this Jolt spec into the field:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "$": "[#2].key",
        "@": "[#2].value"
      }
    }
  }
]

What This Spec Does #

Let's break down the key parts to understand why it works for dynamic fields:

• *: Matches every top-level field in your input JSON, regardless of its name (this is what makes it dynamic—new fields will automatically be included).
• $: References the current field's name, mapping it to the key property of a new array element.
• @: References the current field's value, mapping it to the value property of the same array element.
• [#2]: Jolt's index syntax ensures each key-value pair gets its own entry in the output array. The #2 refers to the level of the wildcard (*), so every field becomes a separate object in the array.

Step 2: Test the Transformation #

For an input like this:

{ 
  "email" : "test@gmail.com", 
  "name" : "somename", 
  "age" : "someage" 
}

The processor will output exactly the format you need:

[
  { "key" : "email", "value": "test@gmail.com" }, 
  { "key" : "name", "value": "somename" }, 
  { "key" : "age", "value": "someage" }
]

Bonus: Handling Nested JSON (If Required) #

If your input has nested fields (e.g., { "user": { "email": "test@gmail.com", "name": "somename" } }), you can adjust the Jolt spec to target specific nested objects:

[
  {
    "operation": "shift",
    "spec": {
      "user": {
        "*": {
          "$": "[#2].key",
          "@": "[#2].value"
        }
      }
    }
  }
]

For deeply nested or arbitrary nested structures, first use a flatten operation in Jolt to collapse nested fields into top-level entries, then apply the original shift spec.

内容的提问来源于stack exchange，提问作者Ragi Dayananda