Typing a dynamically assigned attribute

Question

I am writing a "plugin" for an existing python package and wondering how I would properly type-hint something like this.

Scenario

A python object from a framework I am working with has a plugins attribute that is designed to be extended by user-created plugins. For instance: (very simplified)

# External module that I have no control over

from typing import Any

class BaseClassPluginsCollection:
    ...

class BaseClass:
    def __init__(self):
        self.attr_a = 'a'
        self.plugins = BaseClassPluginsCollection()

    def add_plugin(self, plugin: Any, plugin_name: str):
        setattr(self.plugins, plugin_name, plugin)

So the plugins attribute is basically an empty object which users (like myself) will add new plugin objects to. So in my case, I might do something like this:

class MyCustomPlugin:
    def __init__(self):
        self.my_attr_a = "my attribute"

my_plugin = MyCustomPlugin()
base_object = BaseClass()
base_object.add_plugin(my_plugin, "my_plugin")

In practice, this all works fine and I can add and then use the plugin during program execution without issue. However, I would like to properly type-hint this so that my IDE knows about the plugin. Is such a thing even possible?

Currently (using VS Code) I get the red squiggly line when I try and reference my plugin, even though it does work fine during execution.

For instance, if I try and use the base_object created above, I get an error from the static type checker:

print(f"{base_object.plugins.my_plugin.my_attr_a=}")

What would be the appropriate way to handle this type of scenario? Is there a method for type-hinting something of this sort when its being added during execution using setattr()? Should I be doing some sort if aliasing before trying to access my_plugin?

Answer 1

I am afraid you are out of luck, if you are looking for some magical elegant annotation.

Dynamic attribute assignment

The problem lies in the way that the the package you are using (the one defining BaseClass) is designed. Adding a new attribute to an object using setattr means you are dynamically altering the interface of that object. A static type checker will never be able to pick that up. Even in the simplest case, you will run into errors:

class Foo:
    pass

class Bar:
    pass

f = Foo()
setattr(f, "bar", Bar())
print(type(f.bar))

Even though the output <class '__main__.Bar'> is as expected, mypy immediately complains:

error: "Foo" has no attribute "bar"  [attr-defined]

You might think this is trivial, but consider the situation if the new attribute's name was also dynamically generated by some function func:

setattr(f, func(), Bar())

Now the only way for a type checker to know what even the name of the attribute is supposed to be, is to execute foo.

Static type checkers don't execute your code, they just read it.

So to come back to your code example, there is no way to infer the type of the attribute base_object.plugins.my_plugin because that attribute is set dynamically.

Workaround

What you can always do however is just assure the type checker that my_plugin is in fact of the type you expect it to be. To do that however, you first need to tell the type checker that base_object.plugins even has such an attribute. For that I see no way around subclassing and just declaring that attribute:

class CustomPluginsCollection(BaseClassPluginsCollection):
    my_plugin: MyCustomPlugin


class SubClass(BaseClass):
    plugins: CustomPluginsCollection
    
    def __init__(self) -> None:
        super().__init__()
        self.plugins = CustomPluginsCollection()

my_plugin = MyCustomPlugin()
base_object = SubClass()
base_object.add_plugin(my_plugin, "my_plugin")
print(f"{base_object.plugins.my_plugin.my_attr_a=}")

Now there should be no errors by type checkers and a decent IDE should give you all the type hints for .my_plugin.

Whether or not this is doable in practice depends of course on the details of that framework you are using. It may not be trivial to subclass and (re-)declare the attributes as I did here.

---

As a side note, this is why extensible libraries should try to go the inheritance route, allowing you to subclass their own classes and thus define your own interface. But I don't know enough about the package in question to judge here.