Search code examples
apache-nifiapache-minifi

minifi: listfile processor is not fetching the files from the location

i am just started exploring minifi with nifi, both installed on my local machine. 

minifi: i have listfile -> fetchfile -> RemoteProcessGroup
    RPG:
    url: http://localhost:9090/nifi
    Transport Protocol: HTTP

after i start the minifi, it throws below error in the log file:

 2019-03-13 17:17:59,826 INFO [main] o.apache.nifi.controller.FlowController Not enabling RAW Socket Site-to-Site functionality because nifi.remote.input.socket.port is not set
2019-03-13 17:17:59,833 INFO [main] o.apache.nifi.controller.FlowController Not enabling HTTP(S) Site-to-Site functionality because the 'nifi.remote.input.http.enabled' property is not true
2019-03-13 17:17:59,857 INFO [main] o.a.n.p.FlowConfigurationArchiveManager None of archive max limitation is specified, fall back to the default configuration, maxTime=30 days, maxStorage=500 MB
2019-03-13 17:17:59,951 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 16, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:17:59,958 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 59, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:17:59,975 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 170, col 26: cvc-complex-type.2.4.a: Invalid content was found starting with element 'networkInterface'. One of '{inputPort, outputPort}' is expected.
2019-03-13 17:18:00,056 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 16, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,060 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 59, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,064 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 170, col 26: cvc-complex-type.2.4.a: Invalid content was found starting with element 'networkInterface'. One of '{inputPort, outputPort}' is expected.
2019-03-13 17:18:00,089 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 16, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,091 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 59, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,095 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 170, col 26: cvc-complex-type.2.4.a: Invalid content was found starting with element 'networkInterface'. One of '{inputPort, outputPort}' is expected.
2019-03-13 17:18:00,125 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 16, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,133 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 59, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,142 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 170, col 26: cvc-complex-type.2.4.a: Invalid content was found starting with element 'networkInterface'. One of '{inputPort, outputPort}' is expected.
2019-03-13 17:18:00,423 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 16, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,423 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 59, col 27: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2019-03-13 17:18:00,437 WARN [main] o.a.n.c.StandardFlowSynchronizer Schema validation error parsing Flow Configuration at line 170, col 26: cvc-complex-type.2.4.a: Invalid content was found starting with element 'networkInterface'. One of '{inputPort, outputPort}' is expected.

2019-03-13 17:18:01,296 ERROR [main] o.apache.nifi.controller.FlowController Unable to start ListFile[id=87a17830-7a76-3a03-0000-000000000000] due to java.lang.IllegalStateException: Processor ListFile is not in a valid state due to ['max-performance-metrics' validated against '100000' is invalid because 'max-performance-metrics' is not a supported property or has no Validator associated with it, 'max-listing-time' validated against '3 mins' is invalid because 'max-listing-time' is not a supported property or has no Validator associated with it, 'et-initial-listing-target' validated against 'all' is invalid because 'et-initial-listing-target' is not a supported property or has no Validator associated with it, 'listing-strategy' validated against 'timestamps' is invalid because 'listing-strategy' is not a supported property or has no Validator associated with it, 'track-performance' validated against 'false' is invalid because 'track-performance' is not a supported property or has no Validator associated with it, 'et-time-window' validated against '3 hours' is invalid because 'et-time-window' is not a supported property or has no Validator associated with it, 'et-node-identifier' validated against '${hostname()}' is invalid because 'et-node-identifier' is not a supported property or has no Validator associated with it, 'max-operation-time' validated against '10 secs' is invalid because 'max-operation-time' is not a supported property or has no Validator associated with it]
2019-03-13 17:18:01,313 INFO [Monitor Processor Lifecycle Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled FetchFile[id=6bf93f38-24ad-32cd-0000-000000000000] to run with 1 threads
2019-03-13 17:18:01,330 INFO [main] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled RemoteGroupPort[name=from minifi,targets=http://localhost:9090/nifi] to run with 1 threads

please help on this. thanks in advance

Solution

  • The version of NiFi used to create the flow needs to line up with the version MiNiFi was built against.

    For example, MiNiFi Java 0.5.0 is based on NiFi 1.7.0:

    https://github.com/apache/nifi-minifi/blob/minifi-0.5.0-RC2/pom.xml#L101

    If you use NiFi 1.8.0 or 1.9.0 to create the flow, then the ListFile processor may have new properties that weren't there in 1.7.0 and are therefore unknown to MiNiFi.

    If needed you could manually edit the yaml file and remove the entries for the properties referenced in the ERROR line from the logs.