I am pretty new to Spark. For my project I need to combine data coming from different streams on different ports. To test that I did an exercise which aim is to print data coming from streams from different ports. Below you can see the code:
object hello {
def main(args: Array[String]) {
val ssc = new StreamingContext(new SparkConf(), Seconds(2))
val lines9 = ssc.socketTextStream("localhost", 9999)
val lines8 = ssc.socketTextStream("localhost", 9998)
lines9.print()
lines8.print()
ssc.start()
ssc.awaitTermination()
}
}
Than I run those code and start nc -lk 9999 and nc -lk 9998. When I put anything on port 9999 I see the output on Spark- works fine. When I put anything on 9998 I don't see any output.
Could you please explain me why there is no output on 9998 and how should I implement this to combine those two streams?
Ok, I find an answer for my problem. It is so simple that I feel silly to post it here.
The problem was in executing application. I was doing it by: ./spark-submit --class hello while proper way to do this was: ./spark-submit --class hello --master local[2]
Anyway, thanks guys for contribute.