Search code examples
javajsonapache-nifijolt

How to merge two JSON arrays into a single merged array using Jolt in NiFi


I have a requirement in which I need to merge two JSON arrays into one. Currently the JSON is like this:

[
  {
    "len": 1062,
    "count": 105,
    "info": [
      {
        "name": "test",
        "count": "4"
      },
      {
        "name": "test1",
        "count": "5"
      }
    ],
    "name": "s1"
  },
  {
    "len": 1062,
    "count": 105,
    "info": [
      {
        "name": "test3",
        "count": "8"
      },
      {
        "name": "test4",
        "count": "10"
      }
    ],
    "name": "s2"
  }
]

Now I need to merge the entire JSON array as a single array.

Like:

{
  "len": 1062,
  "count": 105,
  "name": "s1",
  "info": [
    {
      "name": "test",
      "count": "4"
    },
    {
      "name": "test1",
      "count": "5"
    },
    {
      "name": "test3",
      "count": "8"
    },
    {
      "name": "test4",
      "count": "10"
    }
  ]
}

I am new to Jolt and hence don't know what to try


Solution

  • You can use the following shift and cardinality transformations :

    [
      {
        "operation": "shift",
        "spec": {
          "*": {
            "*": "&",
            "info": {
              "*": "&1[]" // &1 replicates the key "test" grabbing the literal's value 
                // after going up the tree one level
            }
          }
        }
      },
      { // pick only the single one from repeating components of the arrays which are generated from
        // the elements other than "info"
        "operation": "cardinality",
        "spec": {
          "*": "ONE",
          "info": "MANY"
        }
      }
    ]
    

    the demo on the site http://jolt-demo.appspot.com/ is :

    enter image description here