I'm running this example provided by apache to stream tweets. However, I can't get any data because it seems that the streaming API is constantly trying to connect to Twitter, thus causing this trace:
2816 [Twitter Stream consumer / [1][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
2927 [Twitter Stream consumer / [2][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
2989 [Twitter Stream consumer / [3][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3063 [Twitter Stream consumer / [4][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3113 [Twitter Stream consumer / [5][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3153 [Twitter Stream consumer / [6][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3197 [Twitter Stream consumer / [7][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3245 [Twitter Stream consumer / [8][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3288 [Twitter Stream consumer / [9][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3333 [Twitter Stream consumer / [10][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3374 [Twitter Stream consumer / [11][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
Popular topics in last 60 seconds (0 total):
3428 [Twitter Stream consumer / [12][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3510 [Twitter Stream consumer / [13][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3563 [Twitter Stream consumer / [14][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
Popular topics in last 10 seconds (0 total):
3595 [Twitter Stream consumer / [15][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3626 [Twitter Stream consumer / [16][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3672 [Twitter Stream consumer / [17][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3723 [Twitter Stream consumer / [18][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3762 [Twitter Stream consumer / [19][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3806 [Twitter Stream consumer / [20][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3849 [Twitter Stream consumer / [21][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3865 [Twitter Stream consumer / [7][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user
3868 [Twitter Stream consumer / [7][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3878 [Twitter Stream consumer / [1][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user
3879 [Twitter Stream consumer / [1][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
3880 [Twitter Stream consumer / [5][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user
3880 [Twitter Stream consumer / [5][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3898 [Twitter Stream consumer / [22][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3911 [Twitter Stream consumer / [3][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Exceeded connection limit for user
3911 [Twitter Stream consumer / [3][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3944 [Twitter Stream consumer / [23][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
3985 [Twitter Stream consumer / [24][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4036 [Twitter Stream consumer / [25][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4086 [Twitter Stream consumer / [26][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4127 [Twitter Stream consumer / [27][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4168 [Twitter Stream consumer / [28][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4208 [Twitter Stream consumer / [29][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4256 [Twitter Stream consumer / [30][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4347 [Twitter Stream consumer / [31][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4377 [Twitter Stream consumer / [4][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.
4377 [Twitter Stream consumer / [4][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4402 [Twitter Stream consumer / [32][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4455 [Twitter Stream consumer / [33][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4504 [Twitter Stream consumer / [34][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4551 [Twitter Stream consumer / [35][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4584 [Twitter Stream consumer / [36][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
Popular topics in last 60 seconds (0 total):
4621 [Twitter Stream consumer / [37][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4664 [Twitter Stream consumer / [38][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
Popular topics in last 10 seconds (0 total):
4697 [Twitter Stream consumer / [39][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4728 [Twitter Stream consumer / [40][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4755 [Twitter Stream consumer / [41][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4786 [Twitter Stream consumer / [42][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4818 [Twitter Stream consumer / [43][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4854 [Twitter Stream consumer / [44][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4877 [Twitter Stream consumer / [8][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.
4877 [Twitter Stream consumer / [8][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4891 [Twitter Stream consumer / [45][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4924 [Twitter Stream consumer / [46][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4955 [Twitter Stream consumer / [47][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
4990 [Twitter Stream consumer / [48][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5025 [Twitter Stream consumer / [49][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5067 [Twitter Stream consumer / [50][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5103 [Twitter Stream consumer / [51][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5137 [Twitter Stream consumer / [52][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5171 [Twitter Stream consumer / [53][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5203 [Twitter Stream consumer / [54][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5235 [Twitter Stream consumer / [55][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5271 [Twitter Stream consumer / [56][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5302 [Twitter Stream consumer / [57][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5334 [Twitter Stream consumer / [58][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5364 [Twitter Stream consumer / [59][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5397 [Twitter Stream consumer / [60][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5411 [Twitter Stream consumer / [9][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.
5411 [Twitter Stream consumer / [9][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5429 [Twitter Stream consumer / [61][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5461 [Twitter Stream consumer / [62][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5492 [Twitter Stream consumer / [63][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5521 [Twitter Stream consumer / [64][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5553 [Twitter Stream consumer / [65][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5582 [Twitter Stream consumer / [66][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5616 [Twitter Stream consumer / [67][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
5650 [Twitter Stream consumer / [68][initializing]]INFO twitter4j.TwitterStreamImpl - Establishing connection.
^CException in thread "dispatcher-event-loop-6" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stop(ReceiverSupervisor.scala:137)
at org.apache.spark.streaming.receiver.ReceiverSupervisorImpl$$anon$1$$anonfun$receive$1.applyOrElse(ReceiverSupervisorImpl.scala:82)
at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:117)
at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:205)
at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:101)
at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:221)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5907 [Twitter Stream consumer / [10][Establishing connection]]INFO twitter4j.TwitterStreamImpl - 420:Returned by the Search and Trends API when you are being rate limited (https://dev.twitter.com/docs/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same account name.
Easy there, Turbo. Too many requests recently. Enhance your calm.
5907 [Twitter Stream consumer / [10][Establishing connection]]INFO twitter4j.TwitterStreamImpl - Waiting for 10000 milliseconds
Exception in thread "receiver-supervisor-future-0" java.lang.AbstractMethodError
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
at org.apache.spark.streaming.twitter.TwitterReceiver.initializeLogIfNecessary(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60)
at org.apache.spark.streaming.twitter.TwitterReceiver.onStop(TwitterInputDStream.scala:104)
at org.apache.spark.streaming.receiver.ReceiverSupervisor.stopReceiver(ReceiverSupervisor.scala:170)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:194)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:189)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Does anyone know why is it trying to establish multiple connections? That seems to be the problem's root cause.
The exception is caused by incompatibilities between the software dependencies. Apache Bahir seems to have ceased development since their last commit was in 2017. Therefore, it seems that the only way to have twitter streaming in spark is to either downgrade everything to compatible versions, or to use twitter4j to create the stream and not the spark streaming constructs.