Search code examples
jsonapache-nifitransformationjolt

Create nested jolt on data with different number of rows


I was trying to convert JSON data into the nested way, making nested on the combination of practice_loc, prac_num, topId.

If practice_loc,prac_num, and topId are the same for two or more data then they will be combined together with different S1, S2,result and additional info within subList. Else they would pass as it is with the addition of subList only.

Data

[
  {
    "practice_loc": 120,
    "prac_num": "oswal",
    "topId": "t1",
    "S1": "A1",
    "Result": "fail",
    "Additional Info": "S1 not matched."
  },
  {
    "practice_loc": "120",
    "prac_num": "oswal",
    "topId": "t1",
    "S1": "A2",
    "S2": "B1",
    "Result": "pass"
  },
  {
    "practice_loc": "334",
    "prac_num": "L3",
    "topId": "plumcherry",
    "S1": "A3",
    "S2": "A1",
    "Result": "fail",
    "Additional Info": "S2 not matched."
  },
  {
    "practice_loc": "987",
    "prac_num": "L3",
    "topId": "artica",
    "Result": "pass"
  }
]

Expected Result is like this will combine the data which has same practice_loc, prac_num, topId.

[
  {
    "practice_loc": "120",
    "prac_num": "oswal",
    "topId": "t1",
    "subList": [
      {
        "S1": "A1",
        "Result": "fail",
        "Additional Info": "S1 not matched."
      },
      {
        "S1": "A2",
        "S2": "B1",
        "Result": "pass"
      }
    ]
  },
  {
    "practice_loc": "334",
    "prac_num": "L3",
    "topId": "plumcherry",
    "subList": [
      {
        "S1": "A3",
        "S2": "A1",
        "Result": "fail",
        "Additional Info": "S2 not matched."
      }
    ]
  },
  {
    "practice_loc": "987",
    "prac_num": "L3",
    "topId": "artica",
    "subList": [
      {
        "Result": "pass"
      }
    ]
  }
]

Solution

  • You can use this spec:

    [
      {
        "operation": "shift",
        "spec": {
          "*": {
            "@": "practice_loc.@(1,practice_loc).prac_num.@(1,prac_num).topId.@(1,topId)[]"
          }
        }
      },
      {
        "operation": "shift",
        "spec": {
          "*": { // practice_loc
            "*": { // value
              "*": { // prac_num
                "*": { // value
                  "*": { // topId
                    "*": { // value
                      "*": { // index of similar objects
                        "practice_loc|prac_num|topId": "&7.&6.&5.&4.&3.&2.&",
                        "*": "&7.&6.&5.&4.&3.&2.subList[&1].&"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      },
      {
        "operation": "shift",
        "spec": {
          "*": {
            "*": {
              "*": {
                "*": {
                  "*": {
                    "*": {
                      "@": ""
                    }
                  }
                }
              }
            }
          }
        }
      },
      {
        "operation": "cardinality",
        "spec": {
          "*": {
            "practice_loc": "ONE",
            "prac_num": "ONE",
            "topId": "ONE"
          }
        }
      }
    ]