How To Use Traverse To Run A List of Effects

Question

I have a function like this:

 def getFile(url: String): EitherT[Future, Error, ByteString]

This is using cats EitherT.

I call this function using another function like this:

def getAllFiles(urls: List[String]): EitherT[Future, Error, List[ByteString]] = {
    urls.map(f => getFile(f).value)
  }

This doesn't work as I get type mismatch:

found   : List[scala.concurrent.Future[Either[Error,akka.util.ByteString]]]
[error]  required: cats.data.EitherT[scala.concurrent.Future,Error,List[akka.util.ByteString]]

No matter what I try I can't this to compile. Basically what I want to do is to run getFile for each URL and download the file in Bytestring.

Answer 1

This works:

import cats.data.EitherT
import cats.effect.IO
import cats.implicits._

type Error = String
type ByteString = Array[Byte]

def getFile(url: String): EitherT[IO, Error, ByteString] = ???

def getAllFiles(urls: List[String]): EitherT[IO, Error, List[ByteString]] =
  urls.traverse(getFile)

But this don't:

def getFile(url: String): EitherT[Future, Error, ByteString] = ???

def getAllFiles(urls: List[String]): EitherT[Future, Error, List[ByteString]] =
  urls.traverse(getFile)

The reason is that traverse expects an Applicative from the nested effect, which cats provides for IO but not for Future.

---

So, the reason is that due the eager and cached nature of Future, we can not really reason about its behavior. So cats doesn't provide instances for it.
You may provide your own Applicative for Future... but, IMHO, it would be easier to just adapt the code using IO.fromFuture and io.unsafeToFuture()