Search code examples
etlapache-nifi

Apache NiFi Transform Epoch Unix Time in Second to Milisecond and Extract the Month and the Year?


I have a JSON like this:

{
   ...
  "timestamp": 1613461012
   ...
}

The timestamp is an epoch / unix timestamp in SECOND not in milisecond.

I would like to have the result like this:

{
       ...
      "timestamp": 1613461012000 //timestamp in ms
      "monthyear": 2021-02 //it can be in any format as long as it can indicate the month and the year from the timestamp
       ...
}

I have tried to get the timestamp in ms by using Jolt Transform like this:

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "timestamp": "=divide(@(1,timestamp), 0.01)"
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "timestamp": "=toInteger(@(1,timestamp))"
    }
  }
]

but it produces 2147483647 as the result not 1613461012000 (I think this can happen because jolt can only use int). For extracting the month and year I haven't found any solution, I think I have to get the epoch in ms first before I can do this.

Any suggestion?


Solution

  • Finally, I can solve this! Here is how I achieve that.

    Full Apache Nifi Schema: Nifi Schema Map

    EvaluateJSONPath Configuration: EvaluateJSONPath

    JoltTransformJSON enter image description here

    JoltSpecification:

    {
        "month.year": "${timestamp:toNumber():multiply(1000):format('yyyy-MM')}",
        "timestamp": ${timestamp:toNumber():multiply(1000)}
    }