Search code examples
jsonapache-nifijolt

Jolt transformer adding current time in long format

Need help with inserting the current date/time in long value using Jolt.

I want the new filed to be inserted for each list element with the current time.

"extracted_at" : ${now()} --> Not working

Input Json

{
  "list": [
    {
      "id": "1",
      "action": "create",
      "actor": {
        "type": "user"
      },
      "audit": {
        "timestamp": 1568677280
      }
    },
    {
      "id": "2",
      "action": "update",
      "actor": {
        "type": "user"
      },
      "audit": {
        "timestamp": 1568677280
      }
    }
  ],
  "response_metadata": {
    "next_cursor": "hmV4dF9jcXJzb3I6MTU3OTg1NTC1jMWFFDGVmZGQwOTU="
  }
}

Output Json

{
  "list": [
    {
      "extracted_at" : 1668677280,
      "id": "1",
      "action": "create",
      "actor": {
        "type": "user"
      },
      "audit": {
        "timestamp": 1568677280
      }
    },
    {
      "extracted_at" : 1668677280,
      "id": "2",
      "action": "update",
      "actor": {
        "type": "user"
      },
      "audit": {
        "timestamp": 1568677280
      }
    }
  ],
  "response_metadata": {
    "next_cursor": "hmV4dF9jcXJzb3I6MTU3OTg1NTC1jMWFFDGVmZGQwOTU="
  }
}

JOLT Spec

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "list": {
        "*": {
          "extractedAt": "${now()}"
        }
      }
    }
  }
]

Nifi Config

enter image description here

Solution

  • As you seem to get unix epoch timestamp in miliseconds, applying a toNumber() conversion to the $now(), which's a date object, will resolve the issue such as

    [
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "list": {
        "*": {
          "extractedAt": "${now():toNumber():divide(1000)}"
        }
      }
    }
  }
]
    • JoltTransformJSON processor in Apache Nifi( current latest version is 1.21.0 ) supports now() as documented
    • Jolt Transform DSL property which should be chain, unlike to the current one, as you already define the operation type within the Specification