Scala stream iterate and memory management

Question

I have this code:

val res = Stream // launch the real computation, which alternates E and M steps, updating the computation state
  .iterate(initCompState)(Base.emIteration)
  .take(nIteration)
  .last

The idea is to provide an initial state initCompState, a function that generates a new state from the previous one, run this for nIterations and get the final result.

I am not interrested in the intermediary states, and would like them to be garbage collected as soon as they are not needed. From what I read online, Stream retains values when they are recursively defined, which is not the case here.

Is my implementation correct and are the intermediate states between the initCompState and res garbage collected as soon as the next state in the stream has been computed ?

Answer 1

Stream is IterableAgain, which means, that it will keep all the elements you iterate through in case you want to see them again.

Iterator is more appropriate in your case, you can scan through it once, so, it discards each element as soon as you look at it. It does not have last though, so, you'd have to implement it yourself with something like

 iterator.foldLeft(Option.empty[Int])((_, n) => Some(n))

Alternatively, just implement your iterations recursively:

 @tailrec
 def iterate(iters: Int, state: State = initCompState): State = if(iters == 0)
   state 
 else 
   iterate(iters - 1, Base.emIteration(state))

 val result = iterate(nIteration)