Search code examples
jsonapache-nifijolt

Jolt Spec to create a counter based on the column


I am facing a problem, transforming a JSON . Requirement is create a counter based on the column.

Input :

[
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "111"
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "111"
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "222"
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "222"
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "222"
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "333"
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "333"
  }
]

In the above JSON, we have 3 different stop IDs. I need to create a stopsequence based on the stopId. If the stopIds are same, i need to retain the counter, if the stopIds are different, counter needs to be incremented.

Output Expected :

[
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "111",
    "stopSequence": 1
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "111",
    "stopSequence": 1
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "222",
    "stopSequence": 2
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "222",
    "stopSequence": 2
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "222",
    "stopSequence": 2
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "333",
    "stopSequence": 3
  },
  {
    "shipment_id": "a60e3760-2f7f-41f3-ac8d-2edd424bcfe4",
    "stopId": "333",
    "stopSequence": 3
  }
]

In the above JSON, we have 3 different stop IDs. I need to create a stopsequence based on the stopId. If the stopIds are same, i need to retain the counter, if the stopIds are different, counter needs to be incremented.


Solution

  • You can manage to solve it through principle of grouping them by "stopid" values such as

    [
      { // group by the "stopId" values
        "operation": "shift",
        "spec": {
          "*": {
            "*": "@1,stopId.&"
          }
        }
      },
      { // get rid of the objects labels
        "operation": "shift",
        "spec": {
          "*": ""
        }
      },
      { // generate ordinals 0,1,2 ... for the objects
        "operation": "shift",
        "spec": {
          "*": "&"
        }
      },
      { // generate the "ss" attribute as constructor for "stopSequence" with values 0,1,2 ...
        "operation": "shift",
        "spec": {
          "*": {
            "@": "&1",
            "$": "&1.ss"
          }
        }
      },
      { // increment the "ss" values by 1
        "operation": "modify-overwrite-beta",
        "spec": {
          "*": {
            "ss": "=intSum(@(1,&),1)"
          }
        }
      },
      { // generate the "stopSequence" arrays
        "operation": "shift",
        "spec": {
          "*": {
            "stopId": {
              "@1": "&2",
              "*": {
                "@2,ss": "&3.stopSequence"
              }
            }
          }
        }
      },
      { // get rid of the "ss" attributes
        "operation": "remove",
        "spec": {
          "*": {
            "ss": ""
          }
        }
      },
      { // flatten all of the arrays
        "operation": "shift",
        "spec": {
          "*": {
            "*": {
              "*": "&2.&.&1"
            }
          }
        }
      },
      { // get rid of the objects labels
        "operation": "shift",
        "spec": {
          "*": {
            "*": ""
          }
        }
      }
    ]
    

    the demo on the site http://jolt-demo.appspot.com/ is :

    enter image description here