JOLT: How to convert datetimestamp to epoch on dynamic json keys

Question

[Edited] I have a use case, where i have to convert the datetimestamp fields to epoch but the fields are not standardized as in some flowfiles we may not receive the datetime fields, My conversion is working perfectly using the EvaluateJsonPath and JoltTransformation JSON.

My Input :

{    
  "updated_on": "2023-03-14 03:56:22",
  "created_on": "2023-03-14 03:56:22"
}

My Jolt Spec :

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "assign": "${assign:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "updated_on": "${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "i_date": "${i_date:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "created_on": "${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
    }
  }
]

Desired Output :

{
  "updated_on": "1678766182",
  "created_on": "1678766182"
}

But i am still not understand why i am getting error whenever any one or two of datetimestamp field missing out of four

Answer 1

Let' reproduce your case;

By adding four processes :

GetFile :

• Input Directory - [defined] ( example : C:\ApacheNifi\App\JSON ) where there's a flowfile.json with flowfile II as the content

• Keep Source File - true

EvaluateJSONPath :

• Destination - flowfile-attribute //to be individually used within the next(Jolt) transformation

  with added properties :

     Property    Value
     ----------- -------------
     created_on  $.created_on
     updated_on  $.updated_on
  

• Path not found behavior - skip => there will be the key name of the attribute while the value is "" (eg. {"created_on":"1678755382000","updated_on":""}) if the related attribute ( in this case updated_on ) doesn't exist within the flowfile.json file.

JoltTransformJSON :

• Specification :

[
  {
    "operation": "modify-overwrite-beta", // replacing "default" with this is important in order to 
                                          // be able to override the values of the existing attributes
    "spec": {      
      "updated_on":"${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "created_on":"${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"      
    }
  }
]       

• LogAttribute

Run once each process except for LogAttribute

As a result, you can get a result similar to the one you've defined as the desired result for the flowfile II from the List queue in the last Connection's context menu.