Search code examples
jsontransformapache-nifijolt

Conversion of data text into JSON data


How to convert by nifi processor

{
  "values": [
    [
      "id",
      "type",
      "name",
      "mobile no"
    ],
    [
      "xyz",
      "detail",
      "bob",
      "5283992123"
    ],
    [
      "pqr",
      "seconDEtail",
      "bob2",
      "6746789887"
    ]
  ]
}

into

[
  {
    "id": "xyz",
    "type": "detail",
    "name": "bob",
    "mobile no ": "5283992123"
  },
  {
    "id": "pqr",
    "type": "seconDEtail",
    "name": "bob2",
    "mobile no ": "6746789887"
  }
]

How to do this conversion by nifi processor in efficient way. The data is dynamic provided by some remote location


Solution

  • You can get the desired output by running two shifts:

    1. Creating temporary object, and grouping the keys and values by position in the array.
    2. Then iterating over that object values to create the outputs
    [
      {
        "operation": "shift",
        "spec": {
          "values": {
            // first element in array
            "0": {
              // for all items in array
              "*": {
                // set value as n.key
                "@": "&.key"
              }
            },
            // for everything else in the array
            "*": {
              // for all items in array
              "*": {
                // set values as n.value[]
                "@": "&.values"
              }
            }
          }
        }
      },
      {
        "operation": "shift",
        "spec": {
          // for all items in our temp object
          "*": {
            "values": {
              // for each values
              "*": {
                // set value with key in our temp object and use index as array position
                "@": "[&1].@(3,key)"
              }
            }
          }
        }
      }
    ]
    

    And to change values of a specific key:

    [
      {
        "operation": "shift",
        "spec": {
          "values": {
            // first element in array
            "0": {
              // for all items in array
              "*": {
                // match id
                "id": {
                  // override value
                  "#transaction_id": "&2.key"
                },
                // for everything else
                "*": {
                  // set value as n.key
                  "@1": "&2.key"
                }
              }
            },
            // for everything else in the array
            "*": {
              // for all items in array
              "*": {
                // set values as n.value[]
                "@": "&.values"
              }
            }
          }
        }
      },
      {
        "operation": "shift",
        "spec": {
          // for all items in our temp object
          "*": {
            "values": {
              // for each values
              "*": {
                // set value with key in our temp object and use index as array position
                "@": "[&1].@(3,key)"
              }
            }
          }
        }
      }
    ]