Search code examples
pythonpython-typingmypy

How would I type-annotate operator `__ior__` on a class inheriting from dict?


I'm defining a specialized dict class. (The specialization details aren't important.)

I'd like to type annotate (type hint) this class, particularly the __ior__ method, but I haven't come up with any annotation that MyPy accepts.

I start with this:

from typing import Any

class MyDict(dict):
    def __ior__(self, other: Any) -> MyDict:
        self.update(other)
        return self

MyPy complains:

main.py:4: error: Signatures of "__ior__" and "__or__" are incompatible  [misc]

So, I define __ior__ and __or__ in MyDict exactly as they're defined in dict, according to reveal_type(dict.__or__) and reveal_type(dict.__ior__):

from typing import overload, Any, Iterable, TypeVar, Union
from _typeshed import SupportsKeysAndGetItem

_KT = TypeVar("_KT")
_VT = TypeVar("_VT")
_T1 = TypeVar("_T1")
_T2 = TypeVar("_T2")

class MyDict(dict[_KT, _VT]):

    def __init__(self, *args: Any, **kwargs: Any):
        ...

    @overload
    def __ior__(self: MyDict[_KT, _VT], other: "SupportsKeysAndGetItem[_KT, _VT]", /) -> MyDict[_KT, _VT]:
        ...

    @overload
    def __ior__(self: MyDict[_KT, _VT], other: Iterable[tuple[_KT, _VT]], /) -> MyDict[_KT, _VT]:
        ...

    @overload
    def __ior__(self: MyDict[_KT, _VT], other: "SupportsKeysAndGetItem[_T1, _T2]", /) -> MyDict[Union[_KT, _T1], Union[_VT, _T2]]:
        ...

    def __ior__(self, other, /):
        self.update(other)
        return self

    @overload
    def __or__(self: MyDict[_KT, _VT], other: dict[_KT, _VT], /) -> MyDict[_KT, _VT]:
        ...

    @overload
    def __or__(self: MyDict[_KT, _VT], other: dict[_T1, _T2], /) -> MyDict[Union[_KT, _T1], Union[_VT, _T2]]:
        ...

    def __or__(self, other, /):
        new = MyDict(self)
        new.update(other)
        return new

That doesn't fix the __ior__/__or__ incompatibility. Moreover, it adds a mismatch between MyDict.__ior__ and dict.__ior__:

main.py:14: error: Signature of "__ior__" incompatible with "__or__" of supertype "dict"  [override]
...

If I add a MyDict.__ior__ overload that complies with the overload of dict.__ior__ that takes a dict[_T1,_T2], it seems to fix the mismatch between MyDict.__ior__ and dict.__or__, but it doesn't fix the __ior__/__or__ incompatiblity.

Why are MyDict's __ior__ and __or__ incompatible? Where would I find the rules for __ior__ and __or__ compatibility?

Here's a link to a mypy playground with my code: https://gist.github.com/mypy-play/97023b90be45766db2844730e59d218c


Solution

  • TL;DR Ignore that diagnostic.

    Let's see what typeshed says about dict.__or__ and dict.__ior__ (here):

    class dict(MutableMapping[_KT, _VT]):
        ...
        if sys.version_info >= (3, 9):
            @overload
            def __or__(self, value: dict[_KT, _VT], /) -> dict[_KT, _VT]: ...
            @overload
            def __or__(self, value: dict[_T1, _T2], /) -> dict[_KT | _T1, _VT | _T2]: ...
    
            @overload
            def __ror__(self, value: dict[_KT, _VT], /) -> dict[_KT, _VT]: ...
            @overload
            def __ror__(self, value: dict[_T1, _T2], /) -> dict[_KT | _T1, _VT | _T2]: ...
    
            # dict.__ior__ should be kept roughly in line with MutableMapping.update()
            @overload  # type: ignore[misc]
            def __ior__(self, value: SupportsKeysAndGetItem[_KT, _VT], /) -> Self: ...
            @overload
            def __ior__(self, value: Iterable[tuple[_KT, _VT]], /) -> Self: ...
    

    Note # type: ignore[misc] for the first overload.

    The diagnostic warns you that x | y and x |= y, where x is a dict, accept different types of y. That's generally a bad sign, however in case of dict this is our reality:

    x: dict[int, int] = {}
    x |= [(1, 2)]  # fine, now x is ``{1: 2}``
    x | [(1, 2)]  # TypeError: unsupported operand type(s) for |: 'dict' and 'list'
    

    So you need to tell the typechecker that it isn't a mistake, you really support different types in those.

    If you want to implement a true type-safe dict subclass, you should use the same overloads as in typeshed or broader (to avoid LSP violation), and silence mypy regarding __or__ vs __ior__ incompatibility.