Search code examples
jsonapache-nifijolt

Transform JSON with array objects into individual records - JOLT spec


I am working on Jolt transform feature in Apache Nifi. I am new to this and need help. The input JSON can have several objects which contain multiple numerical arrays, that need to be flattened.

JSON Input:

[
  {
    "ABC": "aaaaa",
    "DEF": "bbbbb",
    "GHI": "ccccc",
    "JKL": "ddddd",
    "val1": [
      87.6,
      27.42
    ],
    "val2": [
      0.12,
      0.86
    ]
  },
  {
    "ABC": "ppppp",
    "DEF": "qqqqq",
    "GHI": "rrrrr",
    "JKL": "sssss",
    "val1": [
      92.38,
      64.48
    ],
    "val2": [
      1.04,
      0.22
    ]
  },
  ...
]

Expected Output:

[
  {
    "ABC": "aaaaa",
    "DEF": "bbbbb",
    "GHI": "ccccc",
    "JKL": "ddddd",
    "val1": 87.6,
    "val2": 0.12
  },
  {
    "ABC": "aaaaa",
    "DEF": "bbbbb",
    "GHI": "ccccc",
    "JKL": "ddddd",
    "val1": 27.42,
    "val2": 0.86
  },
  {
    "ABC": "ppppp",
    "DEF": "qqqqq",
    "GHI": "rrrrr",
    "JKL": "sssss",
    "val1": 92.38,
    "val2": 1.04
  },
  {
    "ABC": "ppppp",
    "DEF": "qqqqq",
    "GHI": "rrrrr",
    "JKL": "sssss",
    "val1": 64.48
    "val2": 0.22
  }
  ...
]

Please note that root array can have several objects. Can anyone help here? Thanks


Solution

  • You can use the below spec

    [
      {
        "operation": "shift",
        "spec": {
          "*": {
            "val1": {
              "*": {
                "@(2,ABC)": "[#4].&1.ABC",
                "@(2,DEF)": "[#4].&1.DEF",
                "@(2,GHI)": "[#4].&1.GHI",
                "@(2,JKL)": "[#4].&1.JKL",
                "@": "[#4].&1.val1"
              }
            },
            "val2": {
              "*": {
                "@": "[#4].&1.val2"
              }
            }
          }
        }
      },
      {
        "operation": "shift",
        "spec": {
          "*": {
            "*": "[]"
          }
        }
      }
    ]