How to check with mypy that types are not compatible

Imagine I am writing a little Python library with one (nonsense) function:

def takes_a_str(x: str) -> str:
    if x.startswith("."):
        raise RuntimeError("Must not start with '.'")
    return x + ";"

For runtime tests of the functionality, I can check it behaves as expected under both correct conditions (e.g. assert takes_a_str('x')=='x;') and also error conditions (e.g. with pytest.raises(RuntimeError): takes_a_str('.')).

If I want to check that I have not made a mistake with the type hints, I can also perform positive tests: I can create a little test function in a separate file and run mypy or pyright to see that there are no errors:

def check_types() -> None:
    x: str = takes_a_str("")

But I also want to make sure my type hints are not too loose, by checking that some negative cases fail as they ought to:

def should_fail_type_checking() -> None:
    x: dict = takes_a_str("")
    takes_a_str(2)

I can run mypy on this and observe it has errors where I expected, but this is not an automated solution. For example, if I have 20 cases like this, I cannot instantly see that they have all failed, and also may not notice if other errors are nestled amongst them.

Is there a way to ask the type checker to pass, and ONLY pass, where a type conversion does not match? A sort of analogue of pytest.raises() for type checking?

Solution

mypy and pyright both support emitting errors when they detect unnecessary error-suppressing comments. You can utilise this to do an equivalent of pytest.raises, failing a check when things are type-safe. The static type-checking options that need to be turned on are:

mypy: warn_unused_ignores = True / --warn-unused-ignores / strict = True / --strict
pyright: reportUnnecessaryTypeIgnoreComment (see Pyright: Type Check Diagnostics Settings)

Demonstration (mypy Playground, Pyright Playground):

def should_fail_type_checking() -> None:
    # no errors
    x: dict = takes_a_str("")  # type: ignore[assignment] OR # pyright: ignore[reportAssignmentType]
    takes_a_str(2)  # type: ignore[arg-type] OR # pyright: ignore[reportArgumentType]

def check_types() -> None:
    # Failures with mypy and pyright
    # mypy: Unused "type: ignore" comment  [unused-ignore]
    # pyright: Unnecessary "# pyright: ignore" rule: "reportAssignmentType"
    x: str = takes_a_str("")  # type: ignore[assignment] OR # pyright: ignore[reportAssignmentType]

Notes:

The exact error-suppression code should be provided, as if you don't provide specific codes, mypy and pyright will suppress all errors on a line; in your context, this would be equivalent of doing pytest.raises(BaseException).
If you're supporting multiple type checkers (e.g. both mypy and pyright), prefer a mypy diagnostic type: ignore[<mypy error code>] - The comment character sequence # type: ignore... is a Python-typing-compliant code which should be recognised by all type-checkers.

How to check with mypy that types are *not* compatible

How to check with mypy that types are not compatible