Function calls as Python field annotations

I'm working on a small module to make use of annotations to include extra data about class fields by using function calls as annotations (see code below). I am playing with a way to do this while maintaining compatability with static type checking. (Side note: I am doing this with full knowledge of PEP 563 and the postponed evaluation of annotations)

I have run the following code through mypy 0.670 and also pycharm 2019.2.4. mypy reports "error: Invalid type comment or annotation" on the declaration of the value field. However, pycharm infers that the value field is meant to be an int.

pycharm seems to have determined that the result of the function call its_an_int() is the type int and so is able to treat the field as an integer for static type checking and other IDE features. This is ideal and what I would hope that Python type checking can accomplish.

I primarily rely on pycharm and don't use mypy. However, I am cautious about using this design if it is going to conflict with that is considered "sane" for type annotations, and especially if other type checkers like mypy are going to fail on this.

As PEP 563 says, "uses for annotations incompatible with the aforementioned PEPs should be considered deprecated.". I would take this to mean that annotations are mostly meant for indicating types, but I see nothing in any of the PEPs to otherwise discourage the use of expressions in annotations. Presumably, expressions which themselves can be statically analyzed would be acceptable annotations.

Is it reasonable to expect that the value field below can be inferred as an integer by static analysis as currently defined for Python 3.8 (through 4.0)? Is mypy overly strict or limited in its analysis? Or is pycharm being to liberal?

from __future__ import annotations

import typing


def its_an_int() -> typing.Type[int]:
    # ...magic stuff happens here...
    pass


class Foo:

    # This should be as if "value: int" was declared, but with side effects
    # once the annotation is evaluted.
    value: its_an_int()

    def __init__(self, value):
        self.value = value


def y(a: str) -> str:
    return a.upper()


f = Foo(1)

# This call will fail since it is passing an int instead of a string.   A 
# static analyzer should flag the argument type as incorrect if value's type
# is known. 
print(y(f.value))

Solution

It seems unlikely that the syntax you're using will be compliant with type hints as defined by PEP 484.

This is partly because the PEP doesn't ever state that using arbitrary expressions as type hints is allowed, and partly because I don't your example really fits in the spirit of what PEP 484 is trying to accomplish.

In particular, one important design goal of the Python typing ecosystem was to keep a pretty strict divide between the "runtime world" and the "static types" world. In particular, it should always be possible to completely ignore type hints at runtime, but this won't be possible if type hints will sometimes have side-effects when evaluated.

It's not impossible that somebody will eventually design a PEP that allows for what you're trying to do and successfully argue for its acceptance, but I don't think there's anybody working on such a PEP or if there's much demand for one.

Probably the more canonical ways of attaching or recording metadata would probably be to either make the side-effectful operation explicit by doing something like this:

# Alternatively, make this a descriptor class if you want to do
# even fancier things: https://docs.python.org/3/howto/descriptor.html
def magic() -> Any:
    # magic here

class Foo:
    value: int = magic()

    def __init__(self, value):
        self.value = value

...or to use the new Annotated type describe in apparently-just-accepted PEP 593, which allows for type hints and arbitrary non-type-hint information to coexist:

# Note: it should eventually be possible to import directly from 'typing' in
# future versions of Python, but for now you'll need to pip-install
# typing_extensions, the 'typing' backport.
from typing_extensions import Annotated

def magic():
    # magic here

class Foo:
    value: Annotated[int, magic()]

    def __init__(self, value):
        self.value = value

The main caveat with this last approach is that I don't believe Pycharm yet supports the Annotated type hint, given that it's very new.

Setting all this aside, it's worth noting that it's not necessarily wrong to just reject PEP 484 and continue using just whatever Pycharm happens to understand. It's a bit puzzling to me that Pycharm can apparently understand your example (perhaps it's an implementation artifact of how Pycharm is implementing type analysis?), but if it works for you and if adjusting your codebase to be PEP 484 compliant is too painful, it could be reasonable to just roll with what you have.

And if you want to still have your code be available for use by other developers who are using PEP 484 type hints, you could always decide to distribute pyi stub files alongside your package, as described in PEP 561.

It would take a fair amount of work to generate these stub files, but stubs do offer a way to let code that opted out of using PEP 484 to interoperate with code that hasn't.