How can I remove a flowfile or whole object with a null field using JoltTransformJson in Apache NiFi?

Question

How can I transform(remove in this case) flowfile if one of the fields of the object is null using JoltTransformJson processor in nifi

here below is my data.

emp.json:

[
  {
    "Name" : "john Smith",
    "DOB" : "2000-07-14",
    "Salary" : 16000.0
  },
  {
    "Name" : "Sara Jacob",
    "DOB" : "2000-07-14",
    "Salary" : 12000.0
  },
  {
    "Name" : "Peter John",
    "DOB" : null,
    "Salary" : 20000.0
  },
  {
    "Name" : "Lia Poul",
    "DOB" : "2000-07-14",
    "Salary" : 18000.0
  }
]

I used SplitJSON processor to split the JSON File into multiple -separate FlowFiles and i connected the downstream with the JoltTransformJson processor. Next, I configured JoltTransformJson as below: Jolt Transformation DSL: chain Jolt Specification:

[
  {
    "operation": "remove",
    "spec": {
      "rowsToRemove": {
        "*": {
          "DOB": {
            "$null": ""
          }
        }
      }
    }
  }
]

But this expression only removed the DOB field, not the whole object and Flowfile.

Answer 1

Yes you can use JoltTransformJSON processor with the following transformation specs which contain notNull function that pricipally resolves the issue such as

[
  { // produce a new attribute, namely "Nl" to check whether the DOB is null
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "Nl": ["=notNull(@(1,DOB))", "NuLl"]
      }
    }
  },
  { // the following conditional spontaneously removes the object having "Nl" : "NuLl" pair
    "operation": "shift",
    "spec": {
      "*": {
        "Nl": {
          "NuLl": {
            "@2": "@3,Nl"
          },
          "*": {
            "@2": ""
          }
        }
      }
    }
  },
  { // get rid of the remaining "Nl" attributes
    "operation": "remove",
    "spec": {
      "*": {
        "Nl": ""
      }
    }
  }
]

the demo on the site http://jolt-demo.appspot.com/ is