Search code examples
jsonapache-nifijolt

Nifi JOLT Transform to remove JSON Object from JSON Array by element value


I'm new to JSON JOLT transformations and I'm not sure I can perform the function I am attempting. I have a JSON Array that contains JSON Objects. I want to identify a boolean or string value in the objects. If that value is true or equal to something I want to remove the entire object from the array. If the value is false I want to do nothing. Is this possible in NiFi using a JOLT Transformation (JoltTransformJSON Processor).

I've tried a couple different shift's and remove's but not much success with the syntax getting it right. Below is the before and after of what I'm looking to accomplish, if possible. I am fine if what I'm looking to do is not possible, I just don't know enough to determine that yet.

Before:

[
  {
    "alpha": "a",
    "beta": "b",
    "omega": "o",
    "exists": true
  },
  {
    "alpha": "1",
    "beta": "2",
    "omega": "3",
    "exists": false
  },
  {
    "alpha": "z",
    "beta": "x",
    "omega": "y",
    "exists": true
  }
]

Some Transform that looks for exist being false and removes or only pulls out exists that equal true

After:

[
  {
    "alpha": "a",
    "beta": "b",
    "omega": "o",
    "exists": true
  },
  {
    "alpha": "z",
    "beta": "x",
    "omega": "y",
    "exists": true
  }
]

Solution

  • You can set those boolean values as keys of the respective objects such as :

    [
      { // group by the values of the "exists" attribute
        "operation": "shift",
        "spec": {
          "*": {
            "*": "@1,exists.&1.&" // &1 represents going 1 level up the tree 
                                  // to grab the index values for each object 
                                  // within the array
                                  // & is leaf node's value
          }
        }
      },
      { // pick the objects with the "true" key only 
        "operation": "shift",
        "spec": {
          "true": {
            "*": {
              "*": "[#2].&"
            }
          }
        }
      }
    ]
    

    the demo on the site https://jolt-demo.appspot.com/ is :

    enter image description here