Search code examples
apache-sparkapache-nifidataflow

add prefix or sufix in nifi tailFile processor

In my project I am using Nifi to read log file from tomcat and process those data in a spark application , after that insert those process data in DB.

But my problem is that , at app server level , I have 4 tomcat cluster(4 different log file) in 2 different box and I have mark out which data is from which cluster at spark level. In my present set up I have 2 tailFile processor in Nifi which pointing to single outport , in per box but not able to identify which data is from which cluster at spark level.

Is there any option in tailFile processor to add some suffix or prefix or file name(or any attribute) in each record ? so that I can identify each record is coming from which cluster and persist in db in that way?

Any help will be highly appreciated ....

Thanks in advance

Solution

  • The log filename is stored in the tailfile.original.path attribute after TailFile processor

    So, after the TailFile processor place another processor that will add the required prefix to flowfile content . for example you can use ReplaceText.