Search code examples
jsonapache-nifijolt

Jolt - inserting matched field in array list

Below is the input i have that comes from the mergerecord processor where the erp_projectid is the correlation attribute

Input to JOLTTransformJson :

[
  {
    "priority_id": 8,
    "erp_projectid": 1383,
    "erp_subprojectid": 1496,
    "subject": "NewPhaseTest Main"
  },
  {
    "priority_id": 8,
    "erp_projectid": 1383,
    "erp_subprojectid": 1497,
    "subject": "NewPhaseTest-002"
  },
  {
    "erp_projectid": 1383,
    "erp_subprojectid": 1496,
    "project_id": 300
  }
]

Expected Output

[
  {
    "priority_id": 8,
    "erp_projectid": 1383,
    "erp_subprojectid": 1496,
    "subject": "NewPhaseTest Main",
    "project_id": 300
  },
  {
    "priority_id": 8,
    "erp_projectid": 1383,
    "erp_subprojectid": 1497,
    "subject": "NewPhaseTest-002",
    "project_id": 300
  }
]

I've tried writing a jolt spec but it is not giving me the right output ... to give a little bit context there are two executesql that gives me two flow files on which i have passed them through evaluatejsonpath to give them the erp_projectid attribute then i merge them to have one flow file for which the input i have shared and i expect them to pass through jolttransform json to get the desired output also what would be the dsl in that case and if that is not feasible how else i can achieve the output i want?

Solution

  • You can use the following shift transformations :

    [
  { //accumulate the stuff under common "erp_projectid"
    //while prepare the objects without "priority_id" to be eliminated  
    "operation": "shift",
    "spec": {
      "*": {
        "project_id": "@1,erp_projectid.&",
        "*": "@1,erp_projectid.&1.@1,priority_id.&"
      }

    }
  },
  { //transfer the extracted "project_id" attribute
    //into the objects
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "*": {
            "*": "[#3].&",
            "@2,project_id": "[#3].project_id"
          }
        },
        "project_id": { "": "" } //get rid of the transferred attribute
      }
    }
  }
]

    the demo on the site Jolt Transform Demo Using v0.1.1 is :

    the demo on the site

    Edit : if you wanna keep the last object as well, as specified in the comment, then use the following one :

    [
  {
    "operation": "shift",
    "spec": {
      "*": {
        "project_id": "@1,erp_projectid.&",
        "*": "@1,erp_projectid.&1.&" //remove the part ".@1,priority_id" from here
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          // "*": { reduce one layer
          "*": "[#2].&",
          "@1,project_id": "[#2].project_id"
            // }
        },
        "project_id": { "": "" }
      }
    }
  }
]