Search code examples
jsonjolt

JOLT unnesting manipulations


I have a JSON from REST API:

{
  "campaigns": [
    {
      "id": "1",
      "title": "Тест",
      "type": "video",
      "strategy": {
        "name": "cpm",
        "maxMoney": 80000,
        "impressionCost": 0.08,
        "cpm": 80,
        "reachFrequency": 3
      },
      "start": "2021-12-24T14:03:14Z",
      "end": "2022-01-07T20:59:59Z",
      "isArchived": true,
      "publications": []
    },
    {
      "id": "2",
      "title": "RB_SM_Contex_aug2023",
      "type": "article",
      "strategy": {
        "name": "full_read",
        "maxMoney": 100000,
        "maxDailyMoney": 4386,
        "fullReadCost": 6
      },
      "start": "2020-08-12T08:21:18Z",
      "isArchived": true,
      "publications": [
        {
          "id": "11",
          "type": "article",
          "previews": [
            {
              "id": "5f33a7a16b080f7a4d34a9eb",
              "title": "Top 5 movies for last 7 days – Watch on ivi",
              "imageId": "5f359bf13d89681bd2c0a6"
            }
          ],
          "s2sUrlO": "contex ivi"
        },
        {
          "id": "12",
          "type": "article",
          "isDeleted": false,
          "previews": [
            {
              "id": "111",
              "title": "Top 5 movies for last 3 days – Watch on ivi",
              "imageId": "353530951c1e252fadddd0",
              "imageUrl": "https://avatars.mds.yandex.net/get-promoarticles/6191949/-4512463782906749184/orig"
            }
          ],
          "s2sUrlO": "contex ivi"
        },
        {
          "id": "13",
          "type": "article",
          "isDisabled": true,
          "previews": [
            {
              "id": "607d5047eac7203d758cdee7",
              "title": ""
            }
          ]
        }
      ]
    }
  ]
}

I need three different configs to extract all entities from this file: campaigns, publications and previews. I was able to deal with campaigns, but can't handle publications and previews. For publications I want all from publications fields and also add id from campaigns block as campaignId. So I expect to get this result:

[
  {
    "id": "11",
    "type": "article",
    "s2sUrlO": "contex ivi",
    "campaignId": "2"
  },
  {
    "id": "12",
    "type": "article",
    "isDeleted": false,
    "s2sUrlO": "contex ivi",
    "campaignId": "2"
  },
  {
    "id": "13",
    "type": "article",
    "isDisabled": true,
    "campaignId": "2"
  }
]

I wrote next config:

[
  {
    "operation": "shift",
    "spec": {
      "campaigns": {
        "*": {
          "id": "[&1].campaignId",
          "publications": {
            "*": {
              "*": "[&3].&"
            }
          }
        }
      }
    }
  }
]

But it returns not that I expect. It stores publications ids as array and also return rows where publications array was empty - I dont need this. Also, It's okay if it will contain previews array.

And same logic for previews values. But I need to add id from publications array as publicationId, like:

[
  {
    "id":"5f33a7a16b080f7a4d34a9eb",
    "title":"Top 5 movies for last 7 days – Watch on ivi",
    "imageId":"5f359bf13d89681bd2c0a6",
    "publicationId": "5fa7a16b080f7a4d34a9eb"
  },
  ...
]

Solution

  • You can use the following shift tansformation with the desired attributes

    [
      {
        "operation": "shift",
        "spec": {
          "campaigns": {
            "*": {
              "id": "[&1].campaignId", // renaming the attributes
              "publications": {
                "*": {
                  "id|type|s2sUrlO|isD*ed": { // desired attributes
                    "@": "[&2].&1"
                  }
                }
              }
            }
          }
        }
      }
    ]
    

    the demo on the site http://jolt-demo.appspot.com/ is :

    enter image description here