Search code examples
apache-nifi

How do I split comma separrated text file not for one line, but for a several line files?


I am completely new to nifi and I am learning SplitText processor.

So here's the case. I have the comma separated txt file, something like this:

KeyWord, SomeInformation   <---1st line is schema.

KeyWord1, "information"

KeyWord2, "information"

KeyWord1, "another information"

KeyWord2, "another information"

 

and so on.

So the question is how can I split this file on a few files based on KeyWord? So the every line with KeyWord1 go to one file, every line with KeyWord2 go to another file and so on?


Solution

  • Use PartitionRecord and add a user-defined property called keyword (the name is arbitrary but will be the name of the attribute added to each outgoing FlowFile) with value /KeyWord. It will create FlowFiles where one contains all rows whose Keyword field is KeyWord1, another for KeyWord2, and so on.