Search code examples
scalaapache-sparkmapreducerdd

how to get the value from a key,value form a map reduce job in scala


I am trying to get the value from the map-reduce job i.e. 11 from hinein.

(mutuelle,1)
(hinein.,11)
(Pfennig,1)

I'm able to print out all the values, but how do I find only the specific ones?

val counts = books.flatMap(line => line.split(" "))
val MappedRDD = counts.map(word => (word, 1))
val myReducedRDD = MappedRDD.reduceByKey(_ + _)
//myReducedRDD.collect().foreach(println)
val rdd2 = myReducedRDD.map(f => (f))
rdd2.foreach(println)

Solution

  • Have you tried to use filter?

    val myReducedRDD = MappedRDD.reduceByKey(_ + _)
    
    myReducedRDD
      .filter { case (key, value) => key == "hinein" }
      .map { case (key, value) => value } // also can use .map(_._2)
      .foreach(println)
    
    
    // to assign to val
    val result = myReducedRDD
      .filter { case (key, value) => key == "hinein" }
      .map { case (key, value) => value } 
      .collect()(0)