I want to ingest data from twitter with flume in CDH-5.3.2 version. I already config the flume.conf, hbase sink, and twitter sources code.
But, when i start the agent, i got this following error :
2015-06-29 16:57:15,115 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:com.cloudera.flume.source.TwitterSource{name:Twitter,state:IDLE} } - Exception follows.
java.lang.IllegalStateException: Authentication credentials are missing. See http://twitter4j.org/configuration.html for the detail.
at twitter4j.TwitterBaseImpl.ensureAuthorizationEnabled(TwitterBaseImpl.java:200)
at twitter4j.TwitterStreamImpl.sample(TwitterStreamImpl.java:159)
at com.cloudera.flume.source.TwitterSource.start(TwitterSource.java:121)
at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
The flume.conf and hbase sink code were same with this blog : http://ahikmat.blogspot.com/2014/08/streaming-twitter-tweets-to-hbase-with.html
And, the twitter source code was same with https://github.com/cloudera/cdh-twitter-example/tree/master/flume-sources/src/main/java/com/cloudera/flume/source
I don't understand about auth error in twitter4j, whether it was callback url setting / something wrong with Twitter Source code.
You have added your consumer key and secret in the TwitterSourceConstants.java? If you have done this you should reverse this editing. The informations have to be in the flume.conf file. There is an example at twitter_flume. Look in the flume sources folder, there is the flume.conf file.
Hope it helps