Search code examples
pythonpython-typingmypypython-importlib

Typehint importing module dynamically using importlib


Give something as follows:

import importlib

module_path = "mod"
mod = importlib.import_module(module_path, package=None)
print(mod.Foo.Bar.x)

where mod.py is:

class Foo:
    class Bar:
        x = 1

mypy file.py --strict raises the following error:

file.py:7: error: Module has no attribute "Foo"  [attr-defined]

I'm wondering how one is supposed to go about type-hinting this, or if this is something which would typically just be ignored with # type: ignore[attr-defined] (assuming that the code is necessary, and the only options are type-hinting or ignoring the type-hint)?


Why I am using importlib in this situation

The way that importlib is being used is that there's some path:

x.y.<changes>.z

Where <changes> is dynamic, but the others are fixed. I'm confident that the module will contain the attributes which are being called, but due to <changes>, importlib is used for the import.

Which might be summarised as:

I do not know precisely which module I will be importing, but I know it will have a class Foo in it.


Solution

  • As was alluded to by @MisterMiyagi in the comments, I think the solution here is to use structural, rather than nominal, subtyping. Nominal subtyping is where we use direct class inheritance to define type relationships. For example, collections.Counter is a subtype of dict because it directly inherits from dict. Structural subtyping, however, is where we define types based on certain properties a class has or certain behaviours it displays. int is a subtype of typing.SupportsFloat not because it directly inherits from SupportsFloat (it doesn't), but because SupportsFloat is defined as a certain interface, and int satisfies that interface.

    When type-hinting, we can define structural types using typing.Protocol. You could satisfy MyPy in this situation like this:

    import importlib
    from typing import cast, Protocol
    
    class BarProto(Protocol):
        x: int
        
    class FooProto(Protocol):
        Bar: type[BarProto]
        
    class ModProto(Protocol):
        Foo: type[FooProto]
    
    module_path = "mod"
    mod = cast(ModProto, importlib.import_module(module_path, package=None))
    
    print(mod.Foo.Bar.x)
    
    reveal_type(mod)
    reveal_type(mod.Foo)
    reveal_type(mod.Foo.Bar)
    reveal_type(mod.Foo.Bar.x)
    

    We've defined several interfaces here:

    • BarProto: in order to satisfy this interface, a type has to have an attribute x that's of type int.
    • FooProto: in order to satisfy this interface, a type has to have an attribute Bar that is a class of which instances satisfy the BarProto protocol.
    • ModProto: in order to satisfy this interface, a type has to have an attribute Foo that is a class of which instances satisfy the FooProto protocol.

    Then, when importing the module, we use typing.cast to assert to the type-checker that the module we're importing satisfies the ModProto protocol.

    Run it through MyPy, and it informs us it has inferred the following types:

    main.py:18: note: Revealed type is "__main__.ModProto"  
    main.py:19: note: Revealed type is "Type[__main__.FooProto]"
    main.py:20: note: Revealed type is "Type[__main__.BarProto]"
    main.py:21: note: Revealed type is "builtins.int"
    

    Read more about structural subtyping in python here and here.