Search code examples
jsonapache-nifijolt

Root array values without keys


I'd like to bring the "observationDate", "observationId" and "requestedRoutingSkillIds" fields to the root of their respective objects based on this input :

{
  "results": [
    {
      "group": {
        "queueId": "fila1",
        "mediaType": "message"
      },
      "data": [
        {
          "metric": "oInteracting",
          "stats": {
            "count": 1
          },
          "truncated": false,
          "observations": [
            {
              "observationDate": "2023-02-28T17:53:59.933Z",
              "conversationId": "bec3b463-1925-4641-85b2-85618843bee6",
              "requestedRoutingSkillIds": [
                "a283e2f5-800b-451f-ad27-4159695f8e7b"
              ]
            }
          ]
        },
        {
          "metric": "oWaiting",
          "stats": {
            "count": 0
          }
        }
      ]
    },
    {
      "group": {
        "queueId": "fila2",
        "mediaType": "message"
      },
      "data": [
        {
          "metric": "oInteracting",
          "stats": {
            "count": 8
          },
          "truncated": false,
          "observations": [
            {
              "observationDate": "2023-02-28T17:46:56.254Z",
              "conversationId": "25dec92a-415d-407a-be57-e6a7d8cddd2d",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            },
            {
              "observationDate": "2023-02-28T17:49:12.309Z",
              "conversationId": "b40469d5-6cc5-472b-ab72-dafa98a06119",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            },
            {
              "observationDate": "2023-02-28T17:52:23.154Z",
              "conversationId": "4b99b052-5492-4879-a40a-f14865279497",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            },
            {
              "observationDate": "2023-02-28T17:54:34.515Z",
              "conversationId": "a1315187-2c67-4ca0-9165-c560f5e79667",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            },
            {
              "observationDate": "2023-02-28T17:55:52.088Z",
              "conversationId": "8cd531fc-7f21-4f6e-b634-2550de18d385",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            },
            {
              "observationDate": "2023-02-28T17:56:21.913Z",
              "conversationId": "e4bfcdbb-4877-4300-86da-b261d67ee2a0",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            },
            {
              "observationDate": "2023-02-28T17:58:18.176Z",
              "conversationId": "31b1b3a1-1e3e-421f-bb2a-b380f8b67029",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            },
            {
              "observationDate": "2023-02-28T17:58:22.753Z",
              "conversationId": "f0f98ead-cf60-4c96-bdd4-8b4df9caeef3",
              "requestedRoutingSkillIds": [
                "66261478-17c5-4d84-9c57-151e7757f5b2"
              ]
            }
          ]
        },
        {
          "metric": "oWaiting",
          "stats": {
            "count": 0
          }
        }
      ]
    },
    {
      "group": {
        "queueId": "fila3",
        "mediaType": "message"
      },
      "data": [
        {
          "metric": "oInteracting",
          "stats": {
            "count": 1
          },
          "truncated": false,
          "observations": [
            {
              "observationDate": "2023-02-28T17:46:02.033Z",
              "conversationId": "a1596cc7-da60-43bf-b0a8-7ee306ee32a6",
              "requestedRoutingSkillIds": [
                "a4090ade-ff25-4646-9ebe-a6062e251a3f"
              ]
            }
          ]
        },
        {
          "metric": "oWaiting",
          "stats": {
            "count": 0
          }
        }
      ]
    }
  ]
}

I made several attempts looking at more than 20 examples in previous answers but did not get the expected result. The closest maix I got was this one:

Attempt :

[
  {
    "operation": "shift",
    "spec": {
      "results": {
        "*": {
          "@group.queueId": "[&].queueid",
          "@group.mediaType": "[&].mediatype",
          "data": {
            "*": {
              "stats": {
                "@count": "[&4].@2,metric"
              },
              "observations": {
                "*": {
                  "@observationDate": "[&5].observationDate[]",
                  "@conversationId.": "[&5].conversationId[]",
                  "@requestedRoutingSkillIds": "[&5].requestedRoutingSkillIds"
                }
              }
            }
          }
        }
      }
    }
  }
]

Current Output :

[
  {
    "queueid": "fila1",
    "mediatype": "message",
    "oInteracting": 1,
    "observationDate": [
      "2023-02-28T17:53:59.933Z"
    ],
    "conversationId": [
      "bec3b463-1925-4641-85b2-85618843bee6"
    ],
    "requestedRoutingSkillIds": [
      "a283e2f5-800b-451f-ad27-4159695f8e7b"
    ],
    "oWaiting": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message",
    "oInteracting": 8,
    "observationDate": [
      "2023-02-28T17:46:56.254Z",
      "2023-02-28T17:49:12.309Z",
      "2023-02-28T17:52:23.154Z",
      "2023-02-28T17:54:34.515Z",
      "2023-02-28T17:55:52.088Z",
      "2023-02-28T17:56:21.913Z",
      "2023-02-28T17:58:18.176Z",
      "2023-02-28T17:58:22.753Z"
    ],
    "conversationId": [
      "25dec92a-415d-407a-be57-e6a7d8cddd2d",
      "b40469d5-6cc5-472b-ab72-dafa98a06119",
      "4b99b052-5492-4879-a40a-f14865279497",
      "a1315187-2c67-4ca0-9165-c560f5e79667",
      "8cd531fc-7f21-4f6e-b634-2550de18d385",
      "e4bfcdbb-4877-4300-86da-b261d67ee2a0",
      "31b1b3a1-1e3e-421f-bb2a-b380f8b67029",
      "f0f98ead-cf60-4c96-bdd4-8b4df9caeef3"
    ],
    "requestedRoutingSkillIds": [
      "66261478-17c5-4d84-9c57-151e7757f5b2",
      [
        "66261478-17c5-4d84-9c57-151e7757f5b2"
      ],
      [
        "66261478-17c5-4d84-9c57-151e7757f5b2"
      ],
      [
        "66261478-17c5-4d84-9c57-151e7757f5b2"
      ],
      [
        "66261478-17c5-4d84-9c57-151e7757f5b2"
      ],
      [
        "66261478-17c5-4d84-9c57-151e7757f5b2"
      ],
      [
        "66261478-17c5-4d84-9c57-151e7757f5b2"
      ],
      [
        "66261478-17c5-4d84-9c57-151e7757f5b2"
      ]
    ],
    "oWaiting": 0
  },
  {
    "queueid": "fila3",
    "mediatype": "message",
    "oInteracting": 1,
    "observationDate": [
      "2023-02-28T17:46:02.033Z"
    ],
    "conversationId": [
      "a1596cc7-da60-43bf-b0a8-7ee306ee32a6"
    ],
    "requestedRoutingSkillIds": [
      "a4090ade-ff25-4646-9ebe-a6062e251a3f"
    ],
    "oWaiting": 0
  }
]

Expected Output :

[
  {
    "queueid": "fila1",
    "mediatype": "message1",
    "oInteracting1": 1,
    "observationDate": "2023-02-28T17:53:59.933Z",
    "conversationId": "bec3b463-1925-4641-85b2-85618843bee6",
    "requestedRoutingSkillIds": "a283e2f5-800b-451f-ad27-4159695f8e7b",
    "oWaiting1": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "2023-02-28T17:46:56.254Z",
    "conversationId": "25dec92a-415d-407a-be57-e6a7d8cddd2d",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "2023-02-28T17:49:12.309Z",
    "conversationId": "b40469d5-6cc5-472b-ab72-dafa98a06119",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "2023-02-28T17:52:23.154Z",
    "conversationId": "4b99b052-5492-4879-a40a-f14865279497",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "2023-02-28T17:54:34.515Z",
    "conversationId": "a1315187-2c67-4ca0-9165-c560f5e79667",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "2023-02-28T17:55:52.088Z",
    "conversationId": "8cd531fc-7f21-4f6e-b634-2550de18d385",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "2023-02-28T17:56:21.913Z",
    "conversationId": "e4bfcdbb-4877-4300-86da-b261d67ee2a0",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "2023-02-28T17:58:18.176Z",
    "conversationId": "31b1b3a1-1e3e-421f-bb2a-b380f8b67029",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila2",
    "mediatype": "message2",
    "oInteracting2": 8,
    "observationDate": "f0f98ead-cf60-4c96-bdd4-8b4df9caeef3",
    "conversationId": "25dec92a-415d-407a-be57-e6a7d8cddd2d",
    "requestedRoutingSkillIds": "66261478-17c5-4d84-9c57-151e7757f5b2",
    "oWaiting2": 0
  },
  {
    "queueid": "fila3",
    "mediatype": "message3",
    "oInteracting3": 1,
    "observationDate": "2023-02-28T17:58:22.753Z",
    "conversationId": "f0f98ead-cf60-4c96-bdd4-8b4df9caeef3",
    "requestedRoutingSkillIds": "a4090ade-ff25-4646-9ebe-a6062e251a3f",
    "oWaiting3": 0
  }
]

In general, I'm having trouble understanding what each component of the jolt transformation does to create my own solutions (@, &, #, (), [] and their respective combinations).


Solution

  • You can refine desired objects while diving into the "observations" array, and then rearrange the reformed attributes by using shift transformation specs such as

    [
      { // partition by main nodes through sub-indexes of "results" array
        "operation": "shift",
        "spec": {
          "results": {
            "*": {
              "@group.queueId": "&1.queueid",
              "@group.mediaType": "&1.mediatype",
              "data": {
                "*": {
                  "@stats.count": "&3.@1,metric",
                  "observations": {
                    "*": {
                      "*": "&5.&2.&1.&",
                      "requestedR*": {
                        "*": "&6.&3.&2.&1"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      },
      { // accumulate all stuff as looping through "observations" array 
        "operation": "shift",
        "spec": {
          "*": {
            "observations": {
              "*": {
                "@2,queueid": "&3.&1.queueid",
                "@2,mediatype": "&3.&1.mediatype",
                "@2,oInteracting": "&3.&1.oInteracting",
                "@2,oWaiting": "&3.&1.oWaiting",
                "*": "&3.&1.&"
              }
            }
          }
        }
      },
      { // get rid of object keys
        "operation": "shift",
        "spec": {
          "*": {
            "*": ""
          }
        }
      }
    ]
    

    where

    (as an example) "@2,queueId": "&3.&1.queueid" represents

    • going 2 levels up the tree to reach the level of "group" object, which has the same level with "data" array, and grabbing the value of "queueid" attribute

    to match with

    • &3.&1. which represents the values taken from the several sub-indexes by going 3 and 1 levels up the tree respectively

    Btw, single &(without an integer value) on the right hand side means just a replication of the current key