Search code examples
apache-nifi

Merge two JSON flowfile together in NiFi


i want to merge two flowfile that contain JSON object by same specified attribute...

flow1:

attribute:    
xuuid = 123456

content:
{
"sname":"jack",
"id":"00001",
"state":"NY"
}

flow2:

attribute:    
xuuid = 123456

content:
{
"country":"US",
"date":"1983"
}

and i expect this form of data in single output flow:

desired_flow:

attribute:    
xuuid = 123456

content:
{
"sname":"jack",
"id":"00001",
"state":"NY",
"country":"US",
"date":"1983"
}

how do i play with this? MergeContent processor or MergeRecord? i think mergerecord can handle it but i confusing on it.


Solution

  • What you are asking for is a streaming join and it is not something that NiFi really does, similar question and answer here:

    https://stackoverflow.com/a/42909221/5650316

    The merge processors are made to merge pieces of data one after another, not to perform a streaming join. For example, if you have many small json messages you would want to use MergeContent or MergeRecord to merge together thousands of them into a single flow file before writing to HDFS.