Search code examples
jsonapache-nifijolt

How to convert nested data into linear data using jolt


How can I convert this nested data to linear format data as mentioned below using jolt. Need to create separate entry for all the nested data. every record should have 5 data in it practice_loc,prac_num,topId,S1 and S2.

I have to write one jolt for transforming, data below are all possibilities of data

Data

1st case ;

Input :

{
  "practice_loc": 120,
  "prac_num": 234,
  "topId": "t1",
  "subList": [
    {
      "S1": "A1",
      "S2": "B1"
    },
    {
      "S1": "A2"
    }
  ]
}

After transformation:

{
    "practice_loc": 120,
    "prac_num": 234,
    "topId": "t1",
    "S1": "A1",
    "S2": "B1"
  },
  {
    "practice_loc": 120,
    "prac_num": 234,
    "topId": "t1",
    "S1": "A2"
 }

2nd case can be;

Input:

{
  "practice_loc": 987,
  "prac_num": 232,
  "topId": "artica",
  "subList": [
    {
      "S1": "A5",
      "S2": "B7"
    }
  ]
}

After transformation :

{
  "practice_loc": 987,
  "prac_num": 232,
  "topId": "artica",
  "S1": "A5",
  "S2": "B7"
}

3rd case can be;

Input :

{
  "practice_loc": 334,
  "prac_num": 233,
  "topId": "plumcherry",
  "subList": [
    {
      "S1": "A3"
    }
  ]
}

After transformation :

{
  "practice_loc": 334,
  "prac_num": 233,
  "topId": "plumcherry",
  "S1": "A3"
}

4th case can be ;

Input :

{
  "practice_loc": 987,
  "prac_num": 232,
  "topId": "rose",
  "subList": [
    {}
  ]
}

After transformation :

{
  "practice_loc": 987,
  "prac_num": 232,
  "topId": "rose"
}

How can I write only one jolt which can cover all these cases and return the transformed data with whichever type of input it get


Solution

  • You can use the following transformation

    [
      { // group elements by upper level objects wrapper nodes
        "operation": "shift",
        "spec": {
          "subList": {
            "*": {
              "@2|@": "&3_&1" // pick the outer elements as well from the same level of "sublist" through use of @2 while @ returns the value from this array
            }
          }
        }
      },
      { // get rid of the wrappers
        "operation": "shift",
        "spec": {
          "*": {
            "*": {
              "*": "[#3].&"
            }
          }
        }
      },
      { // get rid of the extra generated array, namely "subList"
        "operation": "remove",
        "spec": {
          "*": {
            "subList": ""
          }
        }
      }
    ]