mypy "Optional[Dict[Any, Any]]" is not indexable inside standard filter, map

Given the following code:

from typing import Optional, Dict

def foo(b: bool) -> Optional[Dict]:
    return {} if b else None


def bar() -> None:
    d = foo(False)

    if not d:
        return

    filter(lambda x: d['k'], [])

mypy 0.770 fails with the following error on the last line of bar: Value of type "Optional[Dict[Any, Any]]" is not indexable. Same goes for map. Changing the line to use list comprehension or filter_ or map_ from pydash resolves the error.

Why does mypy throw an error when using the standard filter even though there is a type guard?

Solution

The type-narrowing that happens after an if or assert doesn't propagate down to inner scopes that you've bound that variable in. The easy workaround is to define a new variable bound with the narrower type, e.g.:

def bar() -> None:
    d = foo(False)

    if not d:
        return
    d_exists = d

    filter(lambda x: d_exists['k'], [])

The reason that d isn't bound to the narrower type in the inner scope might be because there's no guarantee that d might not get changed back to None in the outer scope, e.g.:

def bar() -> None:
    d = foo(False)

    if not d:
        return

    def f(x: str) -> str:
        assert d is not None  # this is totally safe, right?
        return d['k']         # mypy passes because of the assert

    d = None  # oh no!
    filter(f, [])

whereas if you bind a new variable, that assignment is not possible:

def bar() -> None:
    d = foo(False)

    if not d:
        return
    d_exists = d

    def f(x: str) -> str:
        # no assert needed because d_exists is not Optional
        return d_exists['k']

    d_exists = None  # error: Incompatible types in assignment
    filter(f, [])

In your particular example there's no runtime danger because the lambda is evaluated immediately by filter with no chance for you to change d in the meantime, but mypy doesn't necessarily have an easy way of determining that the function you called isn't going to hang on to that lambda and evaluate it at a later time.