Jolt to change the key/values into code/values dynamically

Question

I have following input JSON and want to convert these keys/values into codes/values, and need to convert the ts(timestamp) into CET time. These x1,x2,.... keys are dynamic. They can be like y1, y2...., etc and no of keys also dynamic.

Input JSON :

[
  {
    "x1": 29.684,
    "tz_offset": 3600,
    "x2": 29.957,
    "x3": -0.001,
    "x4": 0.041,
    "ts": 1727870442,
    "machine_name": "SL Station"
  }
]

Expected Output JSON :

{
  "Date": "2024-10-02 12:00:42",
  "data": [
    {
      "code": "x1",
      "value": 29.684
    },
    {
      "code": "x2",
      "value": 29.957
    },
    {
      "code": "x3",
      "value": -0.001
    },
    {
      "code": "x4",
      "value": 0.041
    }
  ],
  "machinename": "SL Station"
}

Could you please help in this. Thank you

Answer 1

You can use the processors in this order :

• GetFile

  Assumingly your JSON input is kept as a .json file

  Be careful to set Keep Source File option to true

• EvaluateJsonPath

  

  Destination is set to flowfile-attribute

  Added ts property is set to $[0].ts to be processed within the upcoming processor(eg.JoltTransformJSON)

• JoltTransformJSON

  Jolt Transform DSL option should be chain with the following

  Jolt Specification :

  [
    {//date conversion from unix epoch to standard timestamp
      "operation": "modify-overwrite-beta",
      "spec": {
        "*": {
          "ts": "${ts:multiply(1000):format('yyyy-MM-dd HH:mm:ss', 'CET')}"
        }
      }
    },
    { //get rid of the "tz_offset" attribute 
      "operation": "remove",
      "spec": {
        "*": {
          "tz_offset": ""
        }
      }
    },
    {
      "operation": "shift",
      "spec": {
        "*": {
          "ts": "Date", //rename the attribute
          "*": {
            "$": "data[#2].code",
            "@": "data[#2].value"
          },
          "m*_*": "m&(0,1)&(0,2)" //convert to "machinename", eg. without underscore
        }
      }
    }
  ]
  

whole picture would be like this :