Search code examples
pythongenericsmypypython-typing

mypy error with union of callable and callable generator and typevar

def decorator(
        wrapped: Union[
            Callable[[], T],
            Callable[[], Generator[T, None, None]]
        ]
) -> Callable[[], T]:
    def wrapper():
        value = wrapped()
        if inspect.isgenerator(value):
            return next(value)
        else:
            return value
    return wrapper


@decorator
def foo() -> Generator[str, None, None]:
    yield "bar"

The above code produces the following error in mypy

error: Argument 1 to "decorator" has incompatible type "Callable[[], Generator[str, None, None]]"; expected "Callable[[], Generator[<nothing>, None, None]]"

Is this a limitation in mypy or am I doing something wrong?

Solution

  • It is a bug indeed. After debugging, I've discovered that if you explicitly set T generic:

    U = Union[int, float, str] # your target types here
T = TypeVar('T', U, None)

    or

    T = TypeVar('T', None, Any)

    It works as expected. If I set Any, it has to be the second parameter, but with union or scalar types it is being tested against the first argument.

    Here's a couple of related issues on their GitHub project:

    Update: some more debugging

    Basically, if we write the TypeVar without specifying Any:

    T = TypeVar('T', Type1, Type2, Type3, ...)

    The expected arguments will substitute T with only the first type Type1.

    error: Argument 1 to "decorator" has incompatible type "Callable[[], Generator[str, None, None]]"; expected "Union[Callable[[], Type1], Callable[[], Generator[Type1, None, None]]]"

    But if you do this:

    T = TypeVar('T', Type1, Type2, Type3, Any)

    It will ignore everything and go to Any straightforwardly, without errors. Placing any amount of Any's:

    T = TypeVar('T', Any, Any, ..., Any, Type1, Type2, Type3)

    Will result in ignoring all the previous Any types, and will check against Type1 again, just like without them at all. But, if you don't have anything in TypeVar definition:

    T = TypeVar('T')

    You receive <nothing> as an expected Generator's output.

    Replying to the comment

    Is there a way to define a typevar that essentially covers everything except Generator? I struggle to find a definition that allows instances of any class but not generators.

    Basically, no. From PEP 484:

    No syntax for listing explicitly raised exceptions is proposed. Currently the only known use case for this feature is documentational, in which case the recommendation is to put this information in a docstring.

    There're plenty of questions about this feature on Stack Overflow as well:

    Even without this bug, you would still receive the same results, as by defining:

    T = TypeVar('T', None, Any)

    , since your TypeVar definition is empty, which logically should be an equivalent to (from docs):

    T = TypeVar('T')  # Can be anything