Search code examples
pythontypingdynamic-typing

How to setup python typing for a type argument


I would like to properly add typing for Python types passed as arguments. E.g. let us assume that we would like to add typing to the following function:

def do_something_based_on_types(
    ...
    type_name: str,
    str_to_type_mapping: Dict[str, Any], # what instead of Any?
):
    ...
    object = str_to_type_mapping[type_name]()
    ...

where we would like to pass a mapping from str to type based on which we would like construct an object of selected class. What is a proper typing for this scenario (instead of Any used in a code sample).


Solution

  • Your example is very dynamic of nature. Since the main purpose of Python type annotations is supporting static analysis, there are limits to what they can provide for us here. However, I do think we can do better than Any!

    (Note: to prevent confusion, I will refer to the variable in your snippet named object as my_object, when I use object, I mean the Python type).

    Using typing.Type

    Since you mention the values in the provided dictionary are supposed to be types that can be initiated, a reasonable type annotation would be either Dict[str, Type[object]] or Dict[str, Type], where the latter is equivalent to Dict[str, Type[Any]]. The first is the most restrictive, and will limit what the type checker will allow you to do on your my_object variable. The second is only marginally better than the plain Any, although it does warn the caller when they accidentally provide an instance instead of a type.

    Or typing.Callable?

    From the snippet you provided, the only way the dictionary value is used is to create a new instance. A type is not the only thing that can do this. Actually, any callable (a plain function, a class method) that takes zero arguments and returns an object would suffice as well. So using Callable[[], object] would allow more flexibility to the caller of the function and also warn when a type is passed that doesn't have a zero-argument constructor. I would prefer this over Type as it seems both safer and more in the spirit of Pythons duck-typing.

    Something better than plain object?

    Both solutions have the limitation that inside your function, we don't know anything about the type of my_object. In the general case we are 'forced' to do isinstance checks every time we do an operation not supported by the object type. But, maybe in your use case a single dictionary isn't supposed to contain just any type. It could be that:

    1. All types have a base class in common, say MyBase.
    2. All types are supposed to support a common set of operations (methods or attributes).

    In case (1) we can replace the object type with the type MyBase. The type checker should be able that operations supported by MyBase are safe to use, and warn when the provided dictionary contains other types.

    For case (2), typing.Protocol can help us. You can define a custom protocol with the operations you would require and add that one to the type annotation:

    class MyCustomProto(Protocol):
        foo: int
        def bar(self) -> str: ...
    

    Now the type checker should know that my_object should have the integer attribute foo and a method bar that returns a string. Note that Protocol got added to Python 3.8, and is available for previous versions via the typing-extensions package.

    Dict vs. typing.Mapping

    Not directly related to your question, but if you only use the dictionary to read, you should consider replacing the Dict with the Mapping type. This allows the caller to provide any mapping type other than dict subclasses (unlikely to happen in practice). But more importantly, it warns you when accidentally modifying the supplied mapping, which can result in hard to find bugs. My final suggestion would therefor be Mapping[str, Callable[[], <X>], with <X> being one of object, MyCustomProto or MyBase.