MyPy: 'incompatible type' for virtual class inheritance

Question

Demo code

#!/usr/bin/env python3

from abc import ABCMeta, abstractmethod

class Base(metaclass = ABCMeta):
    @classmethod
    def __subclasshook__(cls, subclass):
        return (
            hasattr(subclass, 'x')
        )

    @property
    @abstractmethod
    def x(self) -> float:
        raise NotImplementedError

class Concrete:
    x: float = 1.0

class Application:
    def __init__(self, obj: Base) -> None:
        print(obj.x)

ob = Concrete() 
app = Application(ob)

print(issubclass(Concrete, Base))
print(isinstance(Concrete, Base))
print(type(ob))
print(Concrete.__mro__)

python test_typing.py returns:

1.0
True
False
<class '__main__.Concrete'>
(<class '__main__.Concrete'>, <class 'object'>)

and mypy test_typing.py returns:

test_typing.py:30: error: Argument 1 to "Application" has incompatible type "Concrete"; expected "Base"
Found 1 error in 1 file (checked 1 source

But if i change the line class Concrete: to class Concrete(Base):, i get for python test_typing.py this:

1.0
True
False
<class '__main__.Concrete'>
(<class '__main__.Concrete'>, <class '__main__.Base'>, <class 'object'>)

and for mypy test_typing.py this:

Success: no issues found in 1 source file

If i add to my code this:

reveal_type(Concrete)
reveal_type(Base)

i get in both cases the same results for it from mypy test_typing.py:

test_typing.py:37: note: Revealed type is "def () -> vmc.test_typing.Concrete"
test_typing.py:38: note: Revealed type is "def () -> vmc.test_typing.Base"

Conclusion

Seems obvious, that MyPi have some problems with virtual base classes but non-virtual inheritance seems working as expected.

Question

How works MyPy's type estimation in these cases? Is there an workaround?

2nd Demo code

Using Protocol pattern:

#!/usr/bin/env python3

from abc import abstractmethod
from typing import Protocol, runtime_checkable

@runtime_checkable
class Base(Protocol):
    @property
    def x(self) -> float:
        raise NotImplementedError
    
    @abstractmethod
    def __init__(self, x: float) -> None:
        raise NotImplementedError

    """
    @classmethod
    def test(self) -> None:
        pass
    """

class Concrete:
    x: float = 1.0

class Application:
    def __init__(self, obj: Base) -> None:
        pass

ob = Concrete() 
app = Application(ob)

Pros

• Working with mypy: Success: no issues found in 1 source file
• Working with isinstance(Concrete, Base) : True

Cons

• Not working with issubclass(Concrete, Base): TypeError: Protocols with non-method members don't support issubclass()
• Not checking the __init__ method signatures: __init__(self, x: float) -> None vs. __init__(self) -> None (Why returns inspect.signature() the strings (self, *args, **kwargs) and (self, /, *args, **kwargs) here? With class Base: instead of class Base(Protocol): i get (self, x: float) -> None and (self, /, *args, **kwargs))
• ignoring the difference between @abstractmethod and @classmethod (treats ANY method as abstract)

3rd Demo code

This time just an more complex example of the 1st code:

#!/usr/bin/env python3

from abc import ABCMeta, abstractmethod
from inspect import signature

class Base(metaclass = ABCMeta):
    @classmethod
    def __subclasshook__(cls, subclass):
        return (
            hasattr(subclass, 'x') and
            (signature(subclass.__init__) == signature(cls.__init__))
        )

    @property
    @abstractmethod
    def x(self) -> float:
        raise NotImplementedError
    
    @abstractmethod
    def __init__(self, x: float) -> None:
        raise NotImplementedError

    @classmethod
    def test(self) -> None:
        pass

class Concrete:
    x: float = 1.0

    def __init__(self, x: float) -> None:
        pass

class Application:
    def __init__(self, obj: Base) -> None:
        pass

ob = Concrete(1.0) 
app = Application(ob)

Pros

• Working with issubclass(Concrete, Base): True
• Working with isinstance(Concrete, Base): False
• Method signature check also for __init__.

Cons

• Not working with MyPy:

  test_typing.py:42: error: Argument 1 to "Application" has incompatible type "Concrete"; expected "Base"
  Found 1 error in 1 file (checked 1 source file)
  

4th Demo code

In some circumstances the following code might be an possible solution.

#!/usr/bin/env python3

from typing import Protocol, runtime_checkable
from dataclasses import dataclass

@runtime_checkable
class Rotation(Protocol):
    @property
    def x(self) -> float:
        raise NotImplementedError
    
    @property
    def y(self) -> float:
        raise NotImplementedError

    @property
    def z(self) -> float:
        raise NotImplementedError

    @property
    def w(self) -> float:
        raise NotImplementedError

@dataclass
class Quaternion:
    x: float = 0.0
    y: float = 0.0
    z: float = 0.0
    w: float = 1.0

    def conjugate(self) -> 'Quaternion':
        return type(self)(
            x = -self.x,
            y = -self.y,
            z = -self.z,
            w = self.w
        )

class Application:
    def __init__(self, rot: Rotation) -> None:
        print(rot)

q = Quaternion(0.7, 0.0, 0.7, 0.0)
app = Application(q.conjugate())

Pros:

• Auto-generated __init__ method because of @dataclass usage. here: (self, x: float = 0.0, y: float = 0.0, z: float = 0.0, w: float = 1.0) -> None
• Works with isinstance(): True
• Works with mypy: Success: no issues found in 1 source file

Cons:

• You need to hope, that the next developer uses @dataclass along with implementing your interface..
• Not usable for __init__ methods, that are not only taken class attributes.

Tipp: If an forced __init__ method is not required and only want to take care of the attributes, then just omit @dataclass.

5th Demo code

Updated the 4th code to provide more safety, but without implicit __init__ method:

#!/usr/bin/env python3

from abc import abstractmethod
from typing import Protocol, runtime_checkable

@runtime_checkable
class Rotation(Protocol):
    @property
    @abstractmethod
    def x(self) -> float:
        raise NotImplementedError
    
    @property
    @abstractmethod
    def y(self) -> float:
        raise NotImplementedError

    @property
    @abstractmethod
    def z(self) -> float:
        raise NotImplementedError

    @property
    @abstractmethod
    def w(self) -> float:
        raise NotImplementedError

class Quaternion:
    _x: float = 0.0
    _y: float = 0.0
    _z: float = 0.0
    _w: float = 1.0

    @property
    def x(self) -> float:
        return self._x

    @property
    def y(self) -> float:
        return self._y

    @property
    def z(self) -> float:
        return self._z

    @property
    def w(self) -> float:
        return self._w

    def __init__(self, x: float, y: float, z: float, w: float) -> None:
        self._x = float(x)
        self._y = float(y)
        self._z = float(z)
        self._w = float(w)

    def conjugate(self) -> 'Quaternion':
        return type(self)(
            x = -self.x,
            y = -self.y,
            z = -self.z,
            w = self.w
        )

    def __str__(self) -> str:
        return ", ".join(
            (
                str(self._x),
                str(self._y),
                str(self._z),
                str(self._w)
            )
        )

    def __repr__(self) -> str:
        cls = self.__class__
        module = cls.__module__
        return f"{module + '.' if module != '__main__' else ''}{cls.__qualname__}({str(self)})"

class Application:
    def __init__(self, rot: Rotation) -> None:
        print(rot)

q = Quaternion(0.7, 0.0, 0.7, 0.0)
app = Application(q.conjugate())

Current conclusion

The Protocol way is unstable. But the Metaclass way is not checkable, because it's not working with MyPy (because it's not static).

Updated question

Are there any alternative solutions to achieve some type of Interfaces (without class Concrete(Base)) AND make it type-safe (checkable)?

Answer 1

Result

After running some tests and more research i am sure, that the actual problem is the behaviour of Protocol to silently overwrite the defined __init__ method.

Conclusion

Seems logical, since Protocols are not intended to be initiated. But sometimes it's required to define an __init__ method, because in my opinion __init__ methods are also part of the interface of classes and it's objects.

Solution

I found an existing issue about this problem, that seems to confirm my point of view: https://github.com/python/cpython/issues/88970

Fortunately it's already fixed: https://github.com/python/cpython/commit/5f2abae61ec69264b835dcabe2cdabe57b9a990e

But unfortunately, this fix will only be part of Python 3.11 and above.

Currenty is Python 3.10.5 available.

WARNING: Like mentioned in the issue, some static type checkers might behave different in this case. MyPy just ignores the missing __init__ method (tested it, confirmed) BUT Pyright seems to detect and report the missing __init__ method (not tested by me).