I have a file with information about the age of people and the number of friends they have. I'm using Spark RDD's and Scala to calculate the average number of friends by age like this:
val friendsByAge: Array[(Int, Double)] = (
actualData
.mapValues((value: Int) => {(value, 1)})
.reduceByKey((acc: (Int, Int), value: (Int, Int)) => (acc._1 + value._1, acc._2 + value._2))
.mapValues((tuple: (Int, Int)) => tuple._1.toDouble / tuple._2)
.collect()
)
This works. However, if I remove the tuple._1.toDouble
conversion from the last step (where I'm dividing two integers) I get the following error:
type mismatch;
found : Array[(Int, Int)]
required: Array[(Int, Double)]
I thought that, when dividing two integers, if I type that the result is Double
it should return a double. If this is not the case, why is this simple code returning a double when I divide two integers?
scala> val a: Double = 10/5
val a: Double = 2.0
scala> println(a)
2.0
scala> println(10/5)
2
In summary, I know how to solve it using tuple._1.toDouble
but I want to know what's going on. I'm new to Scala and I have a Python background so I feel like understanding the basic concepts is key. Thanks in advance.
let's dig a bit more on the provided code
scala> val a: Double = 10/5
val a: Double = 2.0
scala> println(a)
2.0
scala> println(10/5)
2
The first one, looks like it solves the problem, but it is not exactly correct. If you do 10 / 3
the result will not be the expected one
scala> val a: Double = 10 / 3
val a: Double = 3.0
scala> val a: Double = 10.toDouble / 3
val a: Double = 3.3333333333333335
scala> val a: Double = 10 / 3.toDouble
val a: Double = 3.3333333333333335
So, when you write a number in scala, the compiler will infer that is an Int (Unless you add a type).
val a = 1 // infers Int
val b = 1: Double // 1 is converted to Double and then `b` infers Double
val c: Double = 1 // 1 is infered as Int and then cast as Double
If you look at the signatures of the method /
from Int
class you will see
def /(x: Byte): Int
def /(x: Short): Int
def /(x: Char): Int
def /(x: Int): Int
def /(x: Long): Long
def /(x: Float): Float
def /(x: Double): Double
So, the method says what is the return type. Remember also that in scala everything is an object and every value has a type. You can also take a look at how operators work in scala