How to typecheck class with method inserted by metaclass in Python?

Question

In the following code some_method has been added by metaclass:

from abc import ABC
from abc import ABCMeta
from typing import Type


def some_method(cls, x: str) -> str:
    return f"result {x}"


class MyMeta(ABCMeta):
    def __new__(mcs, *args, **kwargs):
        cls = super().__new__(mcs, *args, **kwargs)
        cls.some_method = classmethod(some_method)
        return cls


class MyABC(ABC):
    @classmethod
    def some_method(cls, x: str) -> str:
        return x


class MyClassWithSomeMethod(metaclass=MyMeta):
    pass


def call_some_method(cls: Type[MyClassWithSomeMethod]) -> str:
    return cls.some_method("A")


if __name__ == "__main__":
    mc = MyClassWithSomeMethod()
    assert isinstance(mc, MyClassWithSomeMethod)
    assert call_some_method(MyClassWithSomeMethod) == "result A"

However, MyPy is quite expectedly unhappy about it:

minimal_example.py:27: error: "Type[MyClassWithSomeMethod]" has no attribute "some_method"
Found 1 error in 1 file (checked 1 source file)

Is there any elegant way to tell type checker, that the type is really ok? By elegant, I mean I do not need to change these kinds of definitions everywhere:

class MyClassWithSomeMethod(metaclass=MyMeta): ...

Note, that I do not want to go with subclassing (like with MyABC in the code above). That is, my classes are to be defined with metaclass=.

What options are there?

I've also tried Protocol:

from typing import Protocol

class SupportsSomeMethod(Protocol):
    @classmethod
    def some_method(cls, x: str) -> str:
        ...


class MyClassWithSomeMethod(SupportsSomeMethod, metaclass=MyMeta):
    pass


def call_some_method(cls: SupportsSomeMethod) -> str:
    return cls.some_method("A")

But this leads to:

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Answer 1

As is explained in the MyPy documentation, MyPy's support for metaclasses only goes so far:

Mypy does not and cannot understand arbitrary metaclass code.

The issue is that if you monkey-patch a method onto a class in your metaclass's __new__ method, you could be adding anything to your class's definition. This is much too dynamic for Mypy to understand.

However, all is not lost! You have a few options here.

Option 1: Statically define the method as an instance method on the metaclass

Classes are instances of their metaclass, so instance methods on a metaclass work very similarly to classmethods defined in a class. As such, you can rewrite minimal_example.py as follows, and MyPy will be happy:

from abc import ABCMeta
from typing import Type


class MyMeta(ABCMeta):
    def some_method(cls, x: str) -> str:
        return f"result {x}"


class MyClassWithSomeMethod(metaclass=MyMeta):
    pass


def call_some_method(cls: Type[MyClassWithSomeMethod]) -> str:
    return cls.some_method("A")


if __name__ == "__main__":
    mc = MyClassWithSomeMethod()
    assert isinstance(mc, MyClassWithSomeMethod)
    assert call_some_method(MyClassWithSomeMethod) == "result A"

The only big difference between a metaclass instance-method and your average classmethod is that metaclass instance-methods aren't avaliable from instances of the class using the metaclass:

>>> from abc import ABCMeta
>>> class MyMeta(ABCMeta):
...     def some_method(cls, x: str) -> str:
...         return f"result {x}"
...         
>>> class MyClassWithSomeMethod(metaclass=MyMeta):
...     pass
...     
>>> MyClassWithSomeMethod.some_method('foo')
'result foo'
>>> m = MyClassWithSomeMethod()
>>> m.some_method('foo')
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'MyClassWithSomeMethod' object has no attribute 'some_method'
>>> type(m).some_method('foo')
'result foo'

Option 2: Promise MyPy a method exists, without actually defining it

In lots of situations, you'll be using a metaclass because you want to be more dynamic than is possible if you're statically defining methods. For example, you might want to dynamically generate method definitions on the fly and add them to classes that use your metaclass. In these situations, Option 1 won't do at all.

Another option, in these situations, is to "promise" MyPy that a method exists, without actually defining it. You can do this using standard annotations syntax:

from abc import ABCMeta
from typing import Type, Callable


def some_method(cls, x: str) -> str:
    return f"result {x}"


class MyMeta(ABCMeta):
    some_method: Callable[['MyMeta', str], str]
    
    def __new__(mcs, *args, **kwargs):
        cls = super().__new__(mcs, *args, **kwargs)
        cls.some_method = classmethod(some_method)
        return cls


class MyClassWithSomeMethod(metaclass=MyMeta):
    pass


def call_some_method(cls: Type[MyClassWithSomeMethod]) -> str:
    return cls.some_method("A")


if __name__ == "__main__":
    mc = MyClassWithSomeMethod()
    assert isinstance(mc, MyClassWithSomeMethod)
    assert call_some_method(MyClassWithSomeMethod) == "result A"

This passes MyPy fine, and is actually fairly clean. However, there are limitations to this approach, as the full complexities of a callable cannot be expressed using the shorthand typing.Callable syntax.

Option 3: Lie to MyPy

A third option is to lie to MyPy. There are two obvious ways you could do this.

Option 3(a). Lie to MyPy using the typing.TYPE_CHECKING constant

The typing.TYPE_CHECKING constant is always True for static type-checkers, and always False at runtime. So, you can use this constant to feed different definitions of your class to MyPy than the ones you'll use at runtime.

from typing import Type, TYPE_CHECKING
from abc import ABCMeta 

if not TYPE_CHECKING:
    def some_method(cls, x: str) -> str:
        return f"result {x}"


class MyMeta(ABCMeta):
    if TYPE_CHECKING:
        def some_method(cls, x: str) -> str: ...
    else:
        def __new__(mcs, *args, **kwargs):
            cls = super().__new__(mcs, *args, **kwargs)
            cls.some_method = classmethod(some_method)
            return cls

class MyClassWithSomeMethod(metaclass=MyMeta):
    pass

def call_some_method(cls: Type[MyClassWithSomeMethod]) -> str:
    return cls.some_method("A")


if __name__ == "__main__":
    mc = MyClassWithSomeMethod()
    assert isinstance(mc, MyClassWithSomeMethod)
    assert call_some_method(MyClassWithSomeMethod) == "result A"

This passes MyPy. The main disadvantage of this approach is that it's just plain ugly to have if TYPE_CHECKING checks across your code base.

Option 3(b): Lie to MyPy using a .pyi stub file

Another way of lying to MyPy would be to use a .pyi stub file. You could have a minimal_example.py file like this:

from abc import ABCMeta

def some_method(cls, x: str) -> str:
    return f"result {x}"


class MyMeta(ABCMeta):
    def __new__(mcs, *args, **kwargs):
        cls = super().__new__(mcs, *args, **kwargs)
        cls.some_method = classmethod(some_method)
        return cls

And you could have a minimal_example.pyi stub file in the same directory like this:

from abc import ABCMeta


class MyMeta(ABCMeta):
    def some_method(cls, x: str) -> str: ...

If MyPy finds a .py file and a .pyi file in the same directory, it will always ignore the definitions in the .py file in favour of the stubs in the .pyi file. Meanwhile, at runtime, Python does the opposite, ignoring the stubs in the .pyi file entirely in favour of the runtime implementation in the .py file. So, you can be as dynamic as you like at runtime, and MyPy will be none the wiser.

(As you can see, there is no need to replicate the full method definition in your .pyi file. MyPy only needs the signature of these methods, so the convention is simply to fill the body of a function in a .pyi file with a literal ellipsis ....)

This solution is cleaner than using the TYPE_CHECKING constant. However, I would not get carried away with using .pyi files. Use them as little as possible. If you have a class in your .py file that you do not have a copy of in your stub file, MyPy will be completely ignorant of its existence and raise all sorts of false-positive errors. Remember: if you have a .pyi file, MyPy will completely ignore the .py file that has your runtime implementation in it.

Duplicating class definitions in .pyi files goes against DRY, and runs the risk that you will update your runtime definitions in your .py file but forget to update your .pyi file. If possible, you should isolate the code that truly needs a separate .pyi stub into a single, short file. You should then annotate types as normal in the rest of your project, and import the necessary classes from very_dynamic_classes.py as normal when they are required in the rest of your code.