Option generator in for comprehension forces other generators to be Option?

Question

The first for expression is the base example. The 2nd for-expr introduces a slight variation which I thought would yield the same output. But failed with a compilation error. What is the reason and how to fix?

for {
    n <- List(1,2)
    c <- "ABC"
} yield s"$c$n"
//res0: List[String] = List(A1, B1, C1, A2, B2, C2)

for {
    opt <- List(None, Some(1), None, None, Some(2), None)
    n <- opt
    c <- "ABC"
} yield s"$c$n"
//Error:(14, 5) type mismatch;
//found   : scala.collection.immutable.IndexedSeq[String]
//required: Option[?]
//  c <- "ABC"
//     ^

Answer 1

To reply to the title question: yes, the first generator "sets the mood" for the whole for-expression. Note that for-expressions like the ones above are desugared into calls to flatMaps and a final map (plus withFilter for guards).

In your case, the first for-expression is desugared into the following expression:

List(1, 2).flatMap(n => "ABC".map(c => s"$c$n"))

This can work because Predef (which is implicitly imported in every Scala program) provides an implicit conversion of a String to a Seq[Char].

val implicitlyConverted: Seq[Char] = "ABC"

Thus, it type checks an run as planned.

Now let's give a look at how the second for-expressions is desugared:

List(None, Some(1), None, None, Some(2), None).flatMap(opt => opt.flatMap(n => "ABC".map(c => s"$c$n")))

Again, we have the same type error as above, if we break the expression into several lines we may see the issue a little bit better:

List(None, Some(1), None, None, Some(2), None).flatMap {
  opt =>
    opt.flatMap {
      n =>
        "ABC".map {
          c =>
            s"$c$n"
        }
    }
}

This gives us the following error:

<console>:12: error: type mismatch;
 found   : scala.collection.immutable.IndexedSeq[String]
 required: Option[?]
                      "ABC".map {
                                ^

The second flatMap expects an Option[_] as a result, while the map within ("ABC".map(...)) returns an IndexedSeq[String].

So, this was the reason. How do we fix this? The easiest solution probably involves using a guard and forcibly extracting the value within the Option, like this:

for {
  n <- List(None, Some(1), None, None, Some(2), None)
  c <- "ABC" if n.isDefined
} yield s"$c${n.get}"

A more general way to solve this particular issue involves monad transformers, because the root of the issue is that monads do not compose; the issue is very wide and complex and perhaps this reply may give you a more general answer.