I have a homegrown event collection and processing framework which i am looking to potentially replace with Apache Flume 1.2.0 as we are going to start loading data into Hadoop. Since Flume has no HTTP Source, i am writing a custom source with an embedded Tomcat server. I am following the documentation at http://flume.apache.org/FlumeUserGuide.html#custom-source however when i start i get the following exception:
java.lang.IllegalArgumentException: No known runner type for source com.xxx.xxx.flume.TestSource@5219f360 at org.apache.flume.SourceRunner.forSource(SourceRunner.java:54) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:340) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722)
i think i figured it out. The documentation on the Flume website does not mention a custom implementation should implement not the Source interface but either the EventDrivenSource or PollableSource interface.