(I'm rather new to Python's type annotations and mypy, so I'm describing my problem in detail in order to avoid running into an XY problem)
I have two abstract classes that exchange values of an arbitrary but fixed type:
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import Generic, TypeVar
T = TypeVar('T') # result type
class Command(ABC, Generic[T]):
@abstractmethod
def execute(self, runner: Runner[T]) -> T:
raise NotImplementedError()
class Runner(ABC, Generic[T]):
def run(self, command: Command[T]) -> T:
return command.execute(self)
In my implementation of this interface, the Command
subclass needs to access an attribute of my Runner
subclass (imagine that the command can adapt to runners with different capabilities):
class MyCommand(Command[bool]):
def execute(self, runner: Runner[bool]) -> bool:
# Pseudo code to illustrate dependency on runner's attributes
return runner.magic_level > 10
class MyRunner(Runner[bool]):
magic_level: int = 20
This works as expected, but doesn't satisfy mypy:
mypy_sandbox.py:24: error: "Runner[bool]" has no attribute "magic_level" [attr-defined]
Obviously, mypy is correct: the magic_level
attribute is defined in MyRunner
, but not in Runner
(which is the type of the argument to execute
). So the interface is too generic -- a command doesn't need to work with any runner, only with some runners. So let's make Command
generic on a second type var, to capture the supported runner class:
R = TypeVar('R') # runner type
T = TypeVar('T') # result type
class Command(ABC, Generic[T, R]):
@abstractmethod
def execute(self, runner: R) -> T:
raise NotImplementedError()
class Runner(ABC, Generic[T]):
def run(self, command: Command[T, Runner[T]]) -> T:
return command.execute(self)
class MyCommand(Command[bool, MyRunner]):
def execute(self, runner: MyRunner) -> bool:
# Pseudo code to illustrate dependency on runner's attributes
return runner.magic_level > 10
# MyRunner defined as before
This satisfies mypy, but when I try to use the code, mypy complains again:
if __name__ == '__main__':
command = MyCommand()
runner = MyRunner()
print(runner.run(command))
mypy_sandbox.py:35: error: Argument 1 to "run" of "Runner" has incompatible type "MyCommand"; expected "Command[bool, Runner[bool]]" [arg-type]
This time I don't even understand the error: MyCommand
is a subclass of Command[bool, MyRunner]
, and MyRunner
is a subclass of Runner[bool]
, so why is MyCommand
incompatible with Command[bool, Runner[bool]]
?
And if mypy was satisfied I could probably implement a Command
subclass with a Runner
subclass that uses "a different value" for T
(since R
is not tied to T
) without mypy complaining. I tried R = TypeVar('R', bound='Runner[T]')
, but that throws yet another error:
error: Type variable "mypy_sandbox.T" is unbound [valid-type]
How can I type-annotate this so that extensions as described above are possible but still type-checked correctly?
The current annotations are indeed a contradiction:
Runner
allows only Command
s of the form Command[T, Runner[T]]
.execute
method of Command[bool, Runner[bool]]
accepts any Runner[bool]
.execute
method of MyCommand
only accepts any "Runner[bool]
with a magic_level
".Therefore, MyCommand
is not a Command[bool, Runner[bool]]
– it does not accept any "Runner[bool]
without a magic_level
". This forces MyPy to reject the substitution, even though the reason for it happens earlier.
This issue can be solved by parameterising over R
as the self-type of Runner
. This avoids forcing Runner
to parameterise Command
by the baseclass Runner[T]
, and instead parameterises it by the actual subtype of Runner[T]
.
R = TypeVar('R', bound='Runner[Any]')
T = TypeVar('T') # result type
class Command(ABC, Generic[T, R]):
@abstractmethod
def execute(self, runner: R) -> T:
raise NotImplementedError()
# Runner is not generic in R
class Runner(ABC, Generic[T]):
# Runner.run is generic in its owner
def run(self: R, command: Command[T, R]) -> T:
return command.execute(self)