Typing dict mixin class with Mypy

Question

I'm trying to write a small mixin class to somewhat bridge Set and MutableMapping types: I want the mapping types to have ability to receive some objects (bytes), hash them, and store them, so they are accessible by that hash.

Here's a working version of mixing this class with standard dict:

from hashlib import blake2b

class HashingMixin:
    def add(self, content):
        digest = blake2b(content).hexdigest()
        self[digest] = content

class HashingDict(dict, HashingMixin):
    pass

However I can't figure out how to add type annotations.

From https://github.com/python/mypy/issues/1996 it seems the mixin has to subclass abc.ABC and abc.abstractmethod-define all the methods it expects to call, so here's my shot:

import abc
from hashlib import blake2b
from typing import Dict

class HashingMixin(abc.ABC):
    def add(self, content: bytes) -> None:
        digest = blake2b(content).hexdigest()
        self[digest] = content

    @abc.abstractmethod
    def __getitem__(self, key: str) -> bytes:
        raise NotImplementedError

    @abc.abstractmethod
    def __setitem__(self, key: str, content: bytes) -> None:
        raise NotImplementedError


class HashingDict(Dict[str, bytes], HashingMixin):
    pass

Then Mypy complains about the HashingDict definition:

error: Definition of "__getitem__" in base class "dict" is incompatible with definition in base class "HashingMixin"
error: Definition of "__setitem__" in base class "dict" is incompatible with definition in base class "HashingMixin"
error: Definition of "__setitem__" in base class "MutableMapping" is incompatible with definition in base class "HashingMixin"
error: Definition of "__getitem__" in base class "Mapping" is incompatible with definition in base class "HashingMixin"

Revealing types with:

reveal_type(HashingMixin.__getitem__)
reveal_type(HashingDict.__getitem__)

yields:

error: Revealed type is 'def (coup.content.HashingMixin, builtins.str) -> builtins.bytes'
error: Revealed type is 'def (builtins.dict[_KT`1, _VT`2], _KT`1) -> _VT`2'

I don't know what is wrong :(

Answer 1

This appears to be a bug in mypy -- see this TODO in the code mypy uses to analyze the MRO of classes using multiple inheritance. In short, mypy is incorrectly completing ignoring that you've parameterized Dict with concrete values, and is instead analyzing the code as if you were using Dict.

I believe https://github.com/python/mypy/issues/5973 is probably the most relevant issue in the issue tracker: the root cause is the same.

Until that bug is fixed, you can suppress the errors mypy is generating on that line by adding a # type: ignore to whatever line has the errors. So in your case, you could do the following:

import abc
from hashlib import blake2b
from typing import Dict

class HashingMixin(abc.ABC):
    def add(self, content: bytes) -> None:
        digest = blake2b(content).hexdigest()
        self[digest] = content

    @abc.abstractmethod
    def __getitem__(self, key: str) -> bytes:
        raise NotImplementedError

    @abc.abstractmethod
    def __setitem__(self, key: str, content: bytes) -> None:
        raise NotImplementedError


class HashingDict(Dict[str, bytes], HashingMixin):  # type: ignore
    pass

If you decide to take this approach, I recommend also leaving an additional comment documenting why you're suppressing those errors and running mypy with the --warn-unused-ignores flag.

The former is for the benefit of any future readers of your code; the latter will make mypy report a warning whenever it encounters a # type: ignore that is not actually suppressing any errors and so can safely be deleted.

(And of course, you can always take a stab at contributing a fix yourself!)