Search code examples
pythonmypypython-typingpyarrow

Is mypy contradicting itself or is it just me? Mypy is giving an error on a variable but not on the excatc same literal


The following code:

def foo(bar: dict[ int | float , int | float]) -> None:
    pass

foo({1: 1})
bas = {1: 1}
foo(bas)

Triggers the following mypy error:

6: error: Argument 1 to "foo" has incompatible type "dict[int, int]"; expected "dict[int | float, int | float]"  [arg-type] 
  1. The error is in line 6 not in line 4: i.e. using a literal is OK, but not the same value as a variable. Why?
  2. Why is dict[int, int] not compatible with dict[int | float, int | float]

I ran in to this error using pyarrow, in the function Table.replace_schema_metadata

  1. What can I do to get around this error (apart from # type: ignore[arg-type])?
  2. Is this an error in mypy or in pyarrow?

Using Python 3.11.3 and mypy 1.9.0 (and pyarrow 15.0.1)

Your help is appreciated - if for nothing else - just for my curiosity and sanity :-)

Thanks Troels

Edit: For pyarrow context the following code will result in a mypy error:

metadata = table.schema.metadata
assert metadata is not None
metadata[b'my-metadata'] = b'interesting stuff'
table = table.replace_schema_metadata(metadata)

This arises from pyarrow-stubs containing:

class Schema(_Weakrefable):
    metadata: dict[bytes, bytes] | None
    ...
class Table(_PandasConvertible):
    ...   
def replace_schema_metadata(
        self: _Self, metadata: dict[str | bytes, str | bytes] | None = ...
    ) -> _Self: ...

So the datatypes are directly compatible.


Solution

  • The important thing here to understand is how MyPy does type inference. Since you don't specify variable types other than function signature, MyPy has to "guess" what are the types for other variables. This exact case is even described in MyPy docs

    Declared argument types are also used for type context. In this program mypy knows that the empty list [] should have type list[int] based on the declared type of arg in foo:

    def foo(arg: list[int]) -> None:
        print('Items:', ''.join(str(a) for a in arg))
    
    foo([])  # OK
    

    And the most relevant part:

    However, context only works within a single statement. Here mypy requires an annotation for the empty list, since the context would only be available in the following statement:

    def foo(arg: list[int]) -> None:
        print('Items:', ', '.join(arg))
    
    a = []  # Error: Need type annotation for "a"
    foo(a)
    

    And the solution for it - adding a type annotation for the variable:

    a: list[int] = []  # OK
    foo(a)
    

    Or, in your case:

    bas: dict[int|float, int|float] = {1: 1}
    

    If you want to avoid using lengthy type each time, you can create an alias:

    type numberDict = dict[int|float, int|float]
    
    def foo(bar: numberDict) -> None:
        pass
    
    foo({1: 1})
    bas: numberDict = {1: 1}
    foo(bas)
    

    As for why dict[int, int] is not compatible with dict[int | float, int | float] - because dicts are mutable, it would be perfectly valid for the function to modify it, inserting a float somewhere, which breaks typing of variable that is supposed to be dict[int, int]. It's better described in this question (found by mkrieger1).