Search code examples
jsonapache-nifijolt

Flatten a Complex JSON when there are 2 nested arrrays


I am facing a problem, transforming a very complex nested JSON using jolt transformation. Input and output detail is given below.

I was trying to create a jolt spec but not getting it. Could anyone please help me. Details as below:

Input JSON

{
  "key": [
    {
      "date": "27/09/2023"
    }
  ],
  "value": [
    {
      "values": [
        {
          "location": "3005",
          "total_capacity": 24,
          "capacity_flag": "",
          "orders": [
            {
              "order_type": "INIT",
              "consumed_capacity": 0
            },
            {
              "order_type": "PUSH",
              "consumed_capacity": 0
            }
          ]
        },
        {
          "location": "3007",
          "total_capacity": 72,
          "capacity_flag": "",
          "orders": [
            {
              "order_type": "INIT",
              "consumed_capacity": 0
            },
            {
              "order_type": "PUSH",
              "consumed_capacity": 0
            },
            {
              "order_type": "RPLN",
              "consumed_capacity": 0
            }
          ]
        }
      ]
    }
  ]
}

Desired Output

[
  {
    "date": "27/09/2023",
    "location": "3005",
    "total_capacity": 24,
    "capacity_flag": "",
    "order_type": "INIT",
    "consumed_capacity": 0
  },
  {
    "date": "27/09/2023",
    "location": "3005",
    "total_capacity": 24,
    "capacity_flag": "",
    "order_type": "PUSH",
    "consumed_capacity": 0
  },
  {
    "date": "27/09/2023",
    "location": "3007",
    "total_capacity": 72,
    "capacity_flag": "",
    "order_type": "INIT",
    "consumed_capacity": 0
  },
  {
    "date": "27/09/2023",
    "location": "3007",
    "total_capacity": 72,
    "capacity_flag": "",
    "order_type": "PUSH",
    "consumed_capacity": 0
  },
  {
    "date": "27/09/2023",
    "location": "3007",
    "total_capacity": 72,
    "capacity_flag": "",
    "order_type": "RPLN",
    "consumed_capacity": 0
  }
]

Jolt Spec which i tried, but date is not coming properly

[
  {
    "operation": "shift",
    "spec": {
      "value": {
        "*": {
          "values": {
            "*": {
              "orders": {
                "*": {
                  "order_type": "&3.&1.order_type",
                  "consumed_capacity": "&3.&1.consumed_capacity",
                  "@2,location": "&3.&1.location",
                  "@2,total_capacity": "&3.&1.total_capacity",
                  "@2,capacity_flag": "&3.&1.capacity_flag"
                }
              }
            }
          }
        }
      }
    }
  },
  { // get rid of the object keys generated within the previous spec
    "operation": "shift",
    "spec": {
      "*": {
        "*": ""
      }
    }
  }
]

Need help in writing a jolt spec so that we get a flattened Array


Solution

  • You just to have to go up again and pick up the date before including it into your output.

    Here is an idea of how you can achieve this (Note that I assume that you will always have one date)

    [
      {
        "operation": "shift",
        "spec": {
          "value": {
            "*": {
              "values": {
                "*": {
                  "orders": {
                    "*": {
                      "@(6,key[0].date)": "&3.&1.date",
                      "order_type": "&3.&1.order_type",
                      "consumed_capacity": "&3.&1.consumed_capacity",
                      "@2,location": "&3.&1.location",
                      "@2,total_capacity": "&3.&1.total_capacity",
                      "@2,capacity_flag": "&3.&1.capacity_flag"
                    }
                  }
                }
              }
            }
          }
        }
      },
      { // get rid of the object keys generated within the previous spec
        "operation": "shift",
        "spec": {
          "*": {
            "*": ""
          }
        }
      }
    ]