Search code examples
jsonapache-nifijolt

Jolt JSON null issue

I have issue with Jolt transformation of JSON\

The input JSON :

{
  "_id": "6218e53465793fa20ea11524",
  "patientorderitems": [
    {
      "poi_id": "6218e53465793fa20ea1152a",
      "patientorderlogs": [
        {
          "pol_id": "6218e53465793fa20ea1152e",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        },
        {
          "pol_id": "6218e53465793fa20ea1152c",
          "useruid": "61ee4995f16eebb6b7e1c643",
          "modifiedat": "2022-02-25T17:18:28Z"
        }
      ]
    },
    {
      "poi_id": "6218e53465793fa20ea11525",
      "patientorderlogs": [
        {
          "pol_id": "6218e53465793fa20ea11529",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        }
      ]
    },
    {
      "poi_id": "6218e53465793fa20ea11525",
      "patientorderlogs": [
        {
          "pol_id": "6218e53465793fa20ea11529",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        }
      ]
    },
    {
      "poi_id": "6218e53465793fa20ea11525",
      "patientorderlogs": [
        {
          "pol_id": "6218e53465793fa20ea11529",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        },
        {
          "pol_id": "6218e53465793fa20ea11529",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        },
        {
          "pol_id": "6218e53465793fa20ea11529",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        },
        {
          "pol_id": "6218e53465793fa20ea11529",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        },
        {
          "pol_id": "6218e53465793fa20ea11529",
          "useruid": "61ee4995f16eebb6b7e1c644",
          "modifiedat": "2022-02-25T17:18:28Z"
        }
      ]
    }
  ]
}

The JOLT specification:

[
  {
    "operation": "shift",
    "spec": {
      "patientorderitems": {
        "*": {
          "patientorderlogs": {
            "*": {
              "@(4,_id)": "&1.[&3]._id",
              "@(2,poi_id)": "&1.[&3].poi_id",
              "*": "&1.[&3].&" // combine all innermost attributes by "*" wildcard
            }
          }
        }
      }
    }
  },
  {
    //get rid of indexes and wrapper brackets
    "operation": "shift",
    "spec": {
      "*": {
        "*": ""
      }
    }
  }
]

The output:

[ {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea1152a",
  "pol_id" : "6218e53465793fa20ea1152e",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea11525",
  "pol_id" : "6218e53465793fa20ea11529",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea11525",
  "pol_id" : "6218e53465793fa20ea11529",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea11525",
  "pol_id" : "6218e53465793fa20ea11529",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea1152a",
  "pol_id" : "6218e53465793fa20ea1152c",
  "useruid" : "61ee4995f16eebb6b7e1c643",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, null, null, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea11525",
  "pol_id" : "6218e53465793fa20ea11529",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, null, null, null, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea11525",
  "pol_id" : "6218e53465793fa20ea11529",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, null, null, null, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea11525",
  "pol_id" : "6218e53465793fa20ea11529",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
}, null, null, null, {
  "_id" : "6218e53465793fa20ea11524",
  "poi_id" : "6218e53465793fa20ea11525",
  "pol_id" : "6218e53465793fa20ea11529",
  "useruid" : "61ee4995f16eebb6b7e1c644",
  "modifiedat" : "2022-02-25T17:18:28Z"
} ]

What I expected to have array of JSONS with out nulls
Why I am getting nulls the array

is there any change can I do in the jolt specificatio. ....................................................................................................................................................................................................

Solution

  • You can refine more by adding @(3,poi_id) as value (as being right hand side, raise the needed level from 2 to 3) in order to distinguish objects more such as

    [
  {
    "operation": "shift",
    "spec": {
      "patientorderitems": {
        "*": {
          "patientorderlogs": {
            "*": {
              "@(4,_id)": "@(3,poi_id)[&3].&1._id",
              "@(2,poi_id)": "@(3,poi_id)[&3].&1.poi_id",
              "*": "@(3,poi_id)[&3].&1.&"
            }
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "*": ""
        }
      }
    }
  }
]

    in this case you'll also get rid of nulls as adding one extra level( "*": {..}) to the last spec.

    the demo on the site http://jolt-demo.appspot.com/ is

    enter image description here