Search code examples
jsonapache-nifijolt

Jolt transform json array


I need a help in Jolt transform. I have array, that consist from many json objects. (https://jolt-demo.appspot.com/ )

[
  {
    "table_name": "table_vd",
    "scratch_name": "l1_sample_1_1",
    "scratch_order": 1
  },
  {
    "table_name": "table_vd",
    "scratch_name": "l1_sample_1_1",
    "scratch_order": 34
  },
  {
    "table_name": "table_vd",
    "scratch_name": "l1_sample_2",
    "scratch_order": 3
  }
]

What transformation of Jolt needs to be done to get the following result?

{
  "table_name" : "table_vd",
  "data" : [ {
    "scratch_name" : "l1_sample_1_1",
    "scratch_order" : 1
  }, {
    "scratch_name" : "l1_sample_1_1",
    "scratch_order" : 34
  }, {
    "scratch_name" : "l1_sample_2",
    "scratch_order" : 3
  } ]
}

Now i have next Jolt construction:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "table_name": "table_name",
        "*": {
          "@0": "data[0].&1"
        }
      }
    }
  }
]

But the result below is not correct:

{
  "table_name" : [ "table_vd", "table_vd", "table_vd" ],
  "data" : [ {
    "scratch_name" : [ "l1_sample_1_1", "l1_sample_1_1", "l1_sample_2" ],
    "scratch_order" : [ 1, 34, 3 ]
  } ]
}

Solution

  • You started well by using conditional logic through seperating "table_name" and others("*"). Rearrange "*"(the rest) as "*": "data[&1].&", and then pick the first element of the values of "table_name" array through use of cardinality spec such as

    [
      {
        "operation": "shift",
        "spec": {
          "*": {
            "table_name": "&",
            "*": "data[&1].&"
          }
        }
      },
      {
        "operation": "cardinality",
        "spec": {
          "table_name": "ONE"
        }
      }
    ]
    

    enter image description here