Search code examples
scalafor-comprehension

Dynamically create for comprehension of Futures and wait for completion


I have the following code:

// Start async functions
val async1: Future[Seq[Int]] = ...
val async2: Future[Seq[Int]] = ...
val async3: Future[Seq[Int]] = ...

// Wait for completion
(for {
    a1 <- async1
    a2 <- async2
    a3 <- async3
} yield (a1, a2, a3)).map {
   // Use the results
}

I want to improve this to handle a variable amount of async functions (and not necessarily calling each of them every time). What I have done so far is:

// Start the async functions ?
val asyncs: Seq[Future[Seq[Int]] = otherList.filter(x => someCondition).map(x => asyncFunc(x))

// Wait for the functions to finish ?
(for (seqInt <- asyncs) yield seqInt).map {
    case results => // <-- problem here
        // Use the results 
}

The problem I am having is that the results are of type Future[Seq[Int]], but I expected they would be of type (Seq[Int], Seq[Int], Seq[Int]) like in the first snippet.


In the end I would like to do is kickoff a dynamic amount of async functions which all have the same Future return type, wait for them all to finish, then use all of their results together.


Solution

  • Future.sequence is the key part I was missing (thanks for the comment)

    // Create a list of Futures
    val asyncs: Seq[Future[Seq[Int]] = otherList.filter(x => someCondition).map(x => asyncFunc(x))
    
    // Use Future.sequence to to execute them and return a list of sequence of integers
    Future.sequence(asyncs).map{ 
        case results => // Use the results List[Seq[Int]]
    }.recover {
        case error => // Oh no!
    }