Python type annotation for identical, generic function signatures

typing.Callable takes two "arguments": the argument type(s) and the return type. The argument type should be either ..., for arbitrary arguments, or a list of explicit types (e.g., [str, str, int]).

Is there a way of representing Callables that have exactly the same, albeit arbitrary, signatures for generics?

For example, say I wanted a function that took functions and returned a function with the same signature, I could do this if I knew the function signature upfront:

def fn_combinator(*fn:Callable[[Some, Argument, Types], ReturnType]) -> Callable[[Some, Argument, Types], ReturnType]:
    ...

However, I don't know the argument types upfront and I want my combinator to be suitably general. I had hoped that this would work:

ArgT = TypeVar("ArgT")
RetT = TypeVar("RetT")
FunT = Callable[ArgT, RetT]

def fn_combinator(*fn:FunT) -> FunT:
    ...

However, the parser (at least in Python 3.7) doesn't like ArgT in the first position. Is Callable[..., RetT] the best I can do?

Solution

Prior to Python 3.10

If you don't need to change the function signature at all, you should define FuncT as a TypeVar:

FuncT = TypeVar("FuncT", bound=Callable[..., object])

def fn_combinator(*fn: FuncT) -> FuncT:
    ...

Is there a way of representing Callables that have exactly the same, albeit arbitrary, signatures for generics?

Unlike a type alias (e.g.: FuncT = Callable[..., RetT]), TypeVar allows the type checker to infer a dependency between the parameters and the return value, ensuring that the function signatures will be exactly the same.

However, this approach is utterly limited. Using FuncT makes it difficult to properly type the returned function (See this mypy issue).

def fn_combinator(*fn: FuncT) -> FuncT:
    def combined_fn(*args: Any, **kwargs: Any) -> Any:
        ...

    # return combined_fn  # Won't work. `combined_fn` is not guaranteed to be `FuncT`
    return cast(FuncT, combined_fn)

This is the best we can do as of Python 3.7 due to the limitation of Callable introduced in PEP 484.

... only a list of parameter arguments ([A, B, C]) or an ellipsis (signifying "undefined parameters") were acceptable as the first "argument" to typing.Callable. --- PEP 612

Python 3.10+

Fortunately, type annotations for callables becomes more flexible in Python 3.10 with typing.ParamSpec (the so-called "parameter specification variable") and typing.Concatenate proposed in PEP 612. This extends Callable to support annotating more complicated callables.

This means that you will be able to do the following:

P = ParamSpec("P")
RetT = TypeVar("RetT")

def fn_combinator(*fn: Callable[P, RetT]) -> Callable[P, RetT]:
    ...

It also allows us the fully type check the returned callable without using cast:

def fn_combinator(*fn: Callable[P, RetT]) -> Callable[P, RetT]:
    def combined_fn(*args: P.args, **kwargs: P.kwargs) -> RetT:
        ...

    return combined_fn

See the release notes here.