Search code examples
jsonapache-nifitransformationjolt

JOLT Transformation and add a new array to the output


I have the following JSON input that I would like to transform using JOLT:

{
  "data": {
    "tests": [
      {
        "id": 304674,
        "name": "Test Number One",
        "description": "First Test"
      },
      {
        "id": 12345,
        "name": "Test Number Two",
        "description": "Second test"
      }
    ]
  }
}

I would like the output to rename some things and then add a new array called locations, resulting in the following desired output:

{
  "data": {
    "tests": [
      {
        "TestID": 304674,
        "Name": "Test Number One",
        "Description": "First Test",
        "locations": [
          "Paris",
          "London"
        ],
      },
      {
        "id": 12345,
        "name": "Test Number Two",
        "description": "Second test",
        "locations": [
          "Paris",
          "London"
        ],
      }
    ]
  }
}

I have tried the following JOLT spec on apache-nifi but when I run it I only get "Paris" for locations.

[
  {
    "operation": "shift",
    "spec": {
      "data": {
        "tests": {
          "*": {
            "id": "data.tests[&1].TestID",
            "name": "data.tests[&1].Name",
            "description": "data.tests[&1].Description"
          }
        }
      }
    }
  },
  {
    "operation": "modify-default-beta",
    "spec": {
      "data": {
        "tests": {
          "*": {
            "locations": ["Paris", "London"]
          }
        }
      }
    }
  }
]

Solution

  • So far so good attempt. I can add the following points of view :

    • no need to repeat the literals but use ampersand notations such as &2(eg.:going two levels up the tree and grabbing the literal tests), &3(data)
    • &(0,1) to represent the 1st occurence of the asterisk from the levelcurrent(0th)
    • "locations": ["Paris", "London"] might be directly added within the shift transformation(without needing an extra transformation) while renaming the attributes for the current objects

    So, you can use the following spec:

    [
      {
        "operation": "shift",
        "spec": {
          "data": {
            "tests": {
              "*": {
                "id": "&3.&2[&1].TestID",
                "n*": "&3.&2[&1].N&(0,1)",
                "d*": "&3.&2[&1].D&(0,1)",
                "#Paris|#London": "&3.&2[&1].locations"
              }
            }
          }
        }
      }
    ]