Search code examples
scalalazy-evaluationfor-comprehension

When are scala's for-comprehensions lazy?


In Python, I can do something like this:

lazy = ((i,j) for i in range(0,10000) for j in range(0,10000))
sum((1 for i in lazy))

It will take a while, but the memory use is constant.

The same construct in scala:

(for(i<-0 to 10000; j<-i+1 to 10000) yield (i,j)).count((a:(Int,Int)) => true)

After a while, I get a java.lang.OutOfMemoryError, even though it should be evaluated lazily.


Solution

  • Nothing's inherently lazy about Scala's for-comprehension; it's syntactic sugar* which won't change the fact that the combination of your two ranges will be eager.

    If you work with lazy views of your ranges, the result of the comprehension will be lazy too:

    scala> for(i<-(0 to 10000).view; j<-(i+1 to 10000).view) yield (i,j)
    res0: scala.collection.SeqView[(Int, Int),Seq[_]] = SeqViewN(...)
    
    scala> res0.count((a: (Int, Int)) => true)
    res1: Int = 50005000
    

    The laziness here is nothing to do with the for-comprehension, but because when flatMap or map (see below) are called on some type of container, you get back a result in the same type of container. So, the for-comprehension will just preserve the laziness (or lack of) of whatever you put in.


    *for something like:

    (0 to 10000).flatMap(i => (i+1 to 10000).map(j => (i, j)))