Search code examples
etlapache-nifisyncfusionbigdata

Syncfusion Data integration: CRON driven scheduling strategy - @OnScheduled method NullPointerException


I'm testing Syncfusion Data Integration Platform (based on Apache NiFi) and trying to invoke processor by CRON driven scheduling strategy.

I put into "Run Schedule" field this value: 0 19 22 10 7 ? 2017

But when I start processor I immediately get an error:

failed to invoke @OnScheduled method due to java.lang.NullPointerException

That errors happens with all processors which I try to run by CRON driven scheduling strategy. Timer driven strategy works ok. Pure NiFi (without Syncfusion) works ok too.

What is the reason?

From log file:

ERROR [Timer-Driven Process Thread-1] org.apache.hadoop.util.Shell Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

...

ERROR [StandardProcessScheduler Thread-1] o.a.n.p.standard.GenerateFlowFile GenerateFlowFile[id=347af75d-b2d1-4460-3037-a4ca4fcd25fb] GenerateFlowFile[id=347af75d-b2d1-4460-3037-a4ca4fcd25fb] failed to invoke @OnScheduled method due to java.lang.NullPointerException; processor will not be scheduled to run for 30 seconds: java.lang.NullPointerException
2017-07-10 23:17:53,844 ERROR [StandardProcessScheduler Thread-1] o.a.n.p.standard.GenerateFlowFile 
java.lang.NullPointerException: null
  at org.apache.nifi.controller.scheduling.QuartzSchedulingAgent.doSchedule(QuartzSchedulingAgent.java:185) ~[na:na]
  at org.apache.nifi.controller.scheduling.AbstractSchedulingAgent.schedule(AbstractSchedulingAgent.java:46) ~[na:na]
  at org.apache.nifi.controller.scheduling.QuartzSchedulingAgent.schedule(QuartzSchedulingAgent.java:47) ~[na:na]
  at org.apache.nifi.controller.scheduling.StandardProcessScheduler$4.trigger(StandardProcessScheduler.java:311) ~[na:na]
  at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1286) ~[na:na]
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_25]
  at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_25]
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_25]
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_25]
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_25]
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_25]
  at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
2017-07-10 23:17:53,853 ERROR [StandardProcessScheduler Thread-1] o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due to java.lang.NullPointerException
java.lang.NullPointerException: null
  at org.apache.nifi.controller.scheduling.QuartzSchedulingAgent.doSchedule(QuartzSchedulingAgent.java:185) ~[na:na]
  at org.apache.nifi.controller.scheduling.AbstractSchedulingAgent.schedule(AbstractSchedulingAgent.java:46) ~[na:na]
  at org.apache.nifi.controller.scheduling.QuartzSchedulingAgent.schedule(QuartzSchedulingAgent.java:47) ~[na:na]
  at org.apache.nifi.controller.scheduling.StandardProcessScheduler$4.trigger(StandardProcessScheduler.java:311) ~[na:na]
  at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1286) ~[na:na]
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_25]
  at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_25]
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_25]
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_25]
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_25]
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_25]
  at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
2017-07-10 23:17:56,058 INFO [Flow Service Tasks Thread-1] o.a.n.p.FlowConfigurationArchiveManager Removing old archive file .\conf\archive\20170710T231701+0600_flow.xml.gz to reduce storage usage. currentSize=508959
2017-07-10 23:17:56,060 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@591cd567 // Another save pending = false
2017-07-10 23:18:00,358 INFO [Data Integration Web Server-16] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.GenerateFlowFile
2017-07-10 23:18:00,379 WARN [StandardProcessScheduler Thread-1] o.a.n.controller.StandardProcessorNode Failed while shutting down processor GenerateFlowFile[id=347af75d-b2d1-4460-3037-a4ca4fcd25fb]

...


Solution

  • I think your cron expression is invalid that may be root cause for error.

    What would be meaning for your expression in your cron schedule?

    Replace used expression 0 19 22 10 7 ? 2017 with this 0 19 22 10 7 ? * 2017

    You can generate cron expression with help of below website.

    http://www.cronmaker.com/

    I hope that it will help you.

    Thanks