Search code examples
pythonpython-typingmypy

Typechecking dynamically added attributes


When writing project-specific pytest plugins, I often find the Config object useful to attach my own properties. Example:

from _pytest.config import Config


def pytest_configure(config: Config) -> None:
    config.fizz = "buzz"

def pytest_unconfigure(config: Config) -> None:
    print(config.fizz)

Obviously, there's no fizz attribute in _pytest.config.Config class, so running mypy over the above snippet yields

conftest.py:5: error: "Config" has no attribute "fizz"
conftest.py:8: error: "Config" has no attribute "fizz"

(Note that pytest doesn't have a release with type hints yet, so if you want to actually reproduce the error locally, install a fork following the steps in this comment).

Sometimes redefining the class for typechecking can offer a quick help:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from _pytest.config import Config as _Config

    class Config(_Config):
        fizz: str

else:
    from _pytest.config import Config



def pytest_configure(config: Config) -> None:
    config.fizz = "buzz"

def pytest_unconfigure(config: Config) -> None:
    print(config.fizz)

However, aside from cluttering the code, the subclassing workaround is very limited: adding e.g.

from pytest import Session


def pytest_sessionstart(session: Session) -> None:
    session.config.fizz = "buzz"

would force me to also override Session for typechecking.

What is the best way to resolve this? Config is one example, but I usually have several more in each project (project-specific adjustments for test collection/invocation/reporting etc). I could imagine writing my own version of pytest stubs, but then I would need to repeat this for every project, which is very tedious.


Solution

  • One way of doing this would be to contrive to have your Config object define __getattr__ and __setattr__ methods. If those methods are defined in a class, mypy will use those to type check places where you're accessing or setting some undefined attribute.

    For example:

    from typing import Any
    
    class Config:
        def __init__(self) -> None:
            self.always_available = 1
    
        def __getattr__(self, name: str) -> Any: pass
    
        def __setattr__(self, name: str, value: Any) -> None: pass
    
    c = Config()
    
    # Revealed types are 'int' and 'Any' respectively
    reveal_type(c.always_available)
    reveal_type(c.missing_attr)
    
    # The first assignment type checks, but the second doesn't: since
    # 'already_available' is a predefined attr, mypy won't try using
    # `__setattr__`.
    c.dummy = "foo"
    c.always_available = "foo"
    

    If you know for certain your ad-hoc properties will always be strs or something, you could type __getattr__ and __setattr__ to return or accept str instead of Any respectively to get tighter types.

    Unfortunately, you would still need to do the subtyping trick or mess around with making your own stubs -- the only advantage this gives you is that you at least won't have to list out every single custom property you want to set and makes it possible to create something genuinely reusable. This could maybe make the option more palatable to you, not sure.

    Other options you could explore include:

    • Just adding a # type: ignore comment to every line where you use an ad-hoc property. This would be a somewhat precise, if intrusive, way of suppressing the error messages.
    • Type your pytest_configure and pytest_unconfigure so they accept objects of type Any. This would be a somewhat less intrusive way of suppressing the error messages. If you want to minimize the blast radius of using Any, you could maybe confine any logic that wants to use these custom properties to their own dedicated functions and continue using Config everywhere else.
    • Try using casting instead. For example, inside pytest_configure you could do config = cast(MutableConfig, config) where MutableConfig is a class you wrote that subclasses _pytest.Config and defines both __getattr__ and __setattr__. This is maybe a middle ground between the above two approaches.
    • If adding ad-hoc attributes to Config and similar classes is a common kind of thing to do, maybe try convincing the pytest maintainers to include typing-only __getattr__ and __setattr__ definitions in their type hints -- or some other more dedicated way of letting users add these dynamic properties.