Search code examples
filedirectoryapache-nifitail

NiFi - TailFile - multiple files


In the following path, i have the following logs:

When 10MB are reached for metricbeat file, it will go to write on metricbeat.1 and when this file size exceeds also, it will write to metricbeat.2 etc

[root@ metricbeat]# ls -lorth
total 4.1M
-rwxrwxrwx 1 nifi   10 Aug 17 11:17 metricbeat.2
-rwxrwxrwx 1 nifi   10 Aug 17 11:17 metricbeat.1
-rwxrwxrwx 1 nifi 4.1M Aug 17 11:47 metricbeat

In NiFi (no cluster) I want to tail all the files that are stored on path

/logs/metricbeat/

I am using TailFile Processor with the following Configuration:

TailFile NiFi Processor Configuration

But, the main problem is that i am getting the following error:

'File to Tail' is invalid because There is no file to tail. Files must exist when starting this processor.

If i select the "Single Line" it gets the file "metricbeat"

Could you please tell me what i am doing wrong? Or how can i read all the "metricbeat" files from that path?


Solution

  • "Single File" mode does not require the file to exist before starting the processor, while "Multiple Files" mode does - hence the error you see.