Search code examples
jsonapache-nifijolt

Jolt transformation nifi required for this inputs (json array)


This is data that i am passing as input and expected below output.

[
  {
    "data": {
      "Test1": [
        {
          "role": "test",
          "type": "01",
          "id": "12345",
          "fname": "good",
          "lname": "ked",
          "name": "good ked"
        },
        {
          "role": "test2",
          "type": "02",
          "id": "22322",
          "fname": "net",
          "lname": "hjt",
          "name1": "net hjt",
          "company": "iklus"
        }
      ],
      "test3": {
        "store": "wwffd",
        "dep": "science"
      }
    }
  }
]

we are populating the id ,name ,fname,lname based type 01 and role test and id1,fname1,lname1 ,name1, company based on the type 02 and role test 2

output:

[
  {
    "id": "12345",
    "pn": "other"
  },
  {
    "fname": "good",
    "pn": "ln"
  },
  {
    "lname": "ked",
    "pn": "ln"
  },
  {
    "name": "good ked",
    "pn": "ln"
  },
  {
    "id1": "22322",
    "pn": "other"
  },
  {
    "fname1": "net",
    "pn": "ln"
  },
  {
    "lname1": "hjt",
    "pn": "ln"
  },
  {
    "name1": "net hjt",
    "pn": "ln"
  },
  {
    "company1": "iklus",
    "pn": "ln"
  },
  {
    "store": "wwffd",
    "pn": "ln"
  },
  {
    "dep": "science",
    "pn": "ln"
  }
]

can you please review and let us know jolt spec for this transformation?


Solution

  • You can conditionally match key-value pairs in which only the keys with upper index 1 are suffixed such as :

    [
      { // flatten the JSON value
        "operation": "shift",
        "spec": {
          "*": {
            "*": {
              "T*": { // the level of "Test1" node
                "0": {
                  "*": "&2_&1.&"
                },
                "*": {
                  "*": "&2_&1.&&1"
                }
              },
              "t*": { // the level of "test3" node
                "*": "&1.&"
              }
            }
          }
        }
      },
      { // nest each attributes within separate object
        "operation": "shift",
        "spec": {
          "*": {
            "id|id1": {
              "@": "&2_&1.&",
              "#other": "&2_&1.pn"
            },
            "*": {
              "@": "&2_&1.&",
              "#ln": "&2_&1.pn"
            }
          }
        }
      },
      { // get rid of the object keys
        "operation": "shift",
        "spec": {
          "*": ""
        }
      }
    ]
    

    the demo on the site http://jolt-demo.appspot.com/ is :

    enter image description here