Can mypy report on all the data types of a file? how?

Question

I'm exploring creating some IDE-mypy integration. Obviously I can run mypy to find typing errors. I can also use dmypy with inspect to find the type of a particular variable at a particular line and column. I can also use reveal_type() to inspect a particular variable. But how can I can a list of every piece of type information a file? In effect running reveal_type() on everything. This would include a list of every variable and its type for example. Can mypy do this? If so, how?

Answer 1

Yes, this is possible, but I don't think mypy makes it very easy to do this. You'll also have to decide how to report the information.

---

One possible way to (say) make mypy act as if reveal_type was around every expression is to perform an in-place AST transform to wrap all expressions with a reveal_type(<expression>) call before mypy begins any analysis.

The following is a non-complete plugin implementation which will call reveal_type on every expression in a module with the exact name my_module_of_interest. This plugin requires interpreted mypy (uninstall mypy then e.g. pip install mypy --no-binary mypy); this is because most of mypy's core visitor classes are not subclassable under compiled mypy.

The result of an IDE lint is shown below (PyCharm, 11086-mypy plugin):

Running mypy on this module:

$ mypy my_module_of_interest.py 
my_module_of_interest.py:1: note: Revealed type is "Literal['\ndocstring\n']?"
my_module_of_interest.py:5: note: Revealed type is "Literal[3]?"
my_module_of_interest.py:6: note: Revealed type is "Tuple[Literal[1]?, Literal[2]?, Literal[3]?]"
my_module_of_interest.py:6: note: Revealed type is "Literal[1]?"
my_module_of_interest.py:6: note: Revealed type is "Literal[2]?"
my_module_of_interest.py:6: note: Revealed type is "Literal[3]?"
my_module_of_interest.py:8: note: Revealed type is "Tuple[Literal[1]?, Literal[2]?, Literal[3]?, Literal[4]?]"
my_module_of_interest.py:8: note: Revealed type is "Literal[1]?"
my_module_of_interest.py:8: note: Revealed type is "Literal[2]?"
my_module_of_interest.py:8: note: Revealed type is "Literal[3]?"
my_module_of_interest.py:8: note: Revealed type is "Literal[4]?"
my_module_of_interest.py:12: note: Revealed type is "Literal[8]?"
Success: no issues found in 1 source file

mypy.ini

[mypy]
# Relative directory to the plugin module. Must contain the entry
# point `def plugin(version: str) -> type[mypy.plugin.Plugin]`.
plugins = relative/path/to/reveal_type_plugin.py

relative/path/to/reveal_type_plugin.py

from __future__ import annotations

import typing as t

import mypy.nodes
import mypy.plugin
import mypy.treetransform
import mypy.types


if t.TYPE_CHECKING:
    import mypy.options


def plugin(version: str) -> type[mypy.plugin.Plugin]:

    """
    mypy plugin entry
    """

    return RevealTypePlugin


class RevealTypePlugin(mypy.plugin.Plugin):

    """
    mypy plugin which reports `reveal_type` on expressions in a module
    """

    def get_additional_deps(
        self, file: mypy.nodes.MypyFile
    ) -> list[tuple[int, str, int]]:

        """
        Hook in to `get_additional_deps` to transform a module's expressions into
        `reveal_type(<expression>)`.
        """

        # The test module we're using is called `my_module_of_interest`, with a file
        # path of `my_module_of_interest.py`.
        if file.fullname == "my_module_of_interest":
            transformer: _RevealTypeTransformer = _RevealTypeTransformer()
            transformer.test_only = (
                True  # Needed to allow calling node transforms on a module
            )
            file.defs = transformer.mypyfile(file).defs

        return super().get_additional_deps(file)


class _RevealTypeTransformer(mypy.treetransform.TransformVisitor):

    """
    For all expressions except the l-value of assignment statements, transform the
    expression into `reveal_type(<expression>)`.
    """

    # State variable when set when entering and exiting visitation of an assignment
    # statement's l-values
    _allow_reveal_type_wrap: bool = True

    def visit_assignment_stmt(
        self, node: mypy.nodes.AssignmentStmt
    ) -> mypy.nodes.AssignmentStmt:

        """
        Visits assignment statements, disabling expression transformation when visiting
        the statement's l-values.
        """

        self._allow_reveal_type_wrap = False
        new_lvalues: list[mypy.nodes.Lvalue] = self.expressions(node.lvalues)
        self._allow_reveal_type_wrap = True

        # The following steps are to add and tweak `reveal_type`'s reporting context

        # This will recursively add `reveal_type` to compound expressions:
        # a = 1, 2 -> a = reveal_type((reveal_type(1), reveal_type(2)))
        new_rvalue: mypy.nodes.Expression = self.expr(node.rvalue)
        # Shifts the outer `reveal_type(<...>)` node to be at the
        # left-hand-side assignment statement:
        # a                    = (reveal_type(1), reveal_type(2))
        #             vvvvvvv    ^            ^               ^
        # reveal_type(<tuple>)   <tuple>
        new_rvalue.set_line(node.lvalues[0])
        # At this stage, the `reveal_type` expression is at `a`, but the reporting
        # context is at `<tuple>`, which is still on the right-hand-side of the
        # assignment.
        #
        # Wrap the `reveal_type` in another `reveal_type`. This will now report
        # properly:
        # a                                 = (reveal_type(1), reveal_type(2))
        # ^           vvvvvvvvvvvvvvvvvvvv                 ^               ^
        # reveal_type(reveal_type(<tuple>))
        new_rvalue_at_lvalue: mypy.nodes.Expression = self._makeRevealTypeNode(
            new_rvalue
        )

        # The rest of this is the same as `super().visit_assignment_stmt`
        new: mypy.nodes.AssignmentStmt = mypy.nodes.AssignmentStmt(
            new_lvalues,
            new_rvalue_at_lvalue,
            self.optional_type(node.unanalyzed_type),
        )
        new.line = node.line
        new.is_final_def = node.is_final_def
        new.type = self.optional_type(node.type)
        return new

    def expr(self, expr: mypy.nodes.Expression) -> mypy.nodes.Expression:

        """
        Makes a copy of an expression node, wrapping it in `reveal_type(<node>)` if
        allowable
        """

        new_expr: mypy.nodes.Expression = super().expr(expr)
        if self._allow_reveal_type_wrap:
            new_expr = self._makeRevealTypeNode(new_expr)

        return new_expr

    @staticmethod
    def _makeRevealTypeNode(expr: mypy.nodes.Expression) -> mypy.nodes.CallExpr:

        """
        Turns an expression node into a `reveal_type(<node>)`
        """

        callee_ref_node: mypy.nodes.NameExpr = mypy.nodes.NameExpr(name="reveal_type")
        callee_node: mypy.nodes.CallExpr = mypy.nodes.CallExpr(
            callee_ref_node, [expr], [mypy.nodes.ARG_POS], [None]
        )
        callee_node.set_line(expr)
        return callee_node