Search code examples
pythonpython-3.xapplyhigher-order-functionsbuilt-in

Is there a built-in "apply" function (like "lambda f: f()") as there used to be in Python 2?


Looking at this question, I realised that it is kind of awkward to use multiprocessing's Pool.map if you want is to run a list of functions in parallel:

from multiprocessing import Pool

def my_fun1(): return 1
def my_fun2(): return 2
def my_fun3(): return 3

with Pool(3) as p:
   one, two, three = p.map(lambda f: f(), [my_fun1, my_fun2, my_fun3])

I'm not saying it is exactly cryptic, but I think I expected some conventional name for this, even if only within functools or something, similarly to apply/call in JavaScript (yes, I know JavaScript didn't have lambdas at the time those functions were defined, and no, I'm not saying JavaScript is an exemplary programming language, just an example). In fact, I definitely think something like this should be present in operator, but (unless my eyes deceive me) it seems to be absent. I read that in the case of the identity function the resolution was to let people define their own trivial functions, and I understand it better in that case because there are a couple of different variations you may want, but this one feels like a missing bit to me.

EDIT: As pointed out in the comments, Python 2 used to have an apply function for this purpose.


Solution

  • First, let's look at the practical question.

    For any Python from 2.3 on, you can trivially write not just your no-argument apply, but a perfect-forwarding apply, as a one-liner, as explained in the 2.x docs for apply:

    The use of apply() is equivalent to function(*args, **keywords)

    In other words:

    def apply(function, *args, **keywords):
        return function(*args, **keywords)
    

    … or, as an inline lambda:

    lambda f, *a, **k: f(*a, **kw)
    

    Of course the C implementation was a bit faster, but this is almost never relevant.1

    If you're going to be using this more than once, I think defining the function out-of-line and reusing it by name is probably clearer, but the lamdba version is simple and obvious enough (even more so for your no-args use case) that I can't imagine anyone complaining about it.

    Also, notice that this is actually more trivial than identity if you understand what you're doing, not less. With identity, it's ambiguous what you should return with multiple arguments (or keyword arguments), so you have to decide which behavior you want; with apple, there's only one obvious answer, and it's pretty much impossible to get wrong.


    As for the history:

    Python, like JavaScript, originally had no lambda. It's hard to dig up linkable docs for versions before 2.6, and hard to even find them before 2.3, but I think lambda was added in 1.5, and eventually reached the point where it could be used for perfect forwarding around 2.2. Before then, the docs recommended using apply for forwarding, but after that, the docs recommended using lambda in place of apply. In fact, there was no longer any recommended use of apply.

    So in 2.3, the function was deprecated.2

    During the Python-3000 discussions that led to 3.0, Guido suggested that all of the "functional programming" functions except maybe map and filter were unnecessary.3 Others made good cases for reduce and partial.4 But a big part of the case was that they're actually not trivial to write (in fully-general form), and easy to get wrong. That isn't true for apply. Also, people were able to find relevant uses of reduce and partial in real-world codebases, but the only uses of apply anyone could find were old pre-2.3 code. In fact, it was so rare that it wasn't even worth making the 2to3 tool transform calls to apply.

    The final rationale for removing it was summarized in PEP 3100:

    apply(): use f(*args, **kw) instead [2]

    That footnote links to an essay by Guido called "Python Regrets", which is now a 404 link. The accompanying PowerPoint presentation is still available, however, or you can view an HTML flipbook of the presentation he wrote it for. But all it really says is the same one-liner, and IIRC, the only further discussion was "We already effectively got rid of it in 2.3."


    1. In most idiomatic Python code that has to apply a function, the work inside that function is pretty heavy. In your case, of course, the overhead of calling the functions (pickling arguments and passing them over a pipe) is even heavier. The one case where it would matter is when you're doing "Haskell-style functional programming" instead of "Lisp-style"—that is, very few function definitions, and lots of functions made by transforming functions and composing the results. But that's already so slow (and stack-heavy) in Python that it's not a reasonable thing to do. (Flat use of decorators to apply a wrapper or three works great, but a potentially unbounded chain of wrappers will kill your performance.)

    2. The formal deprecation mechanism didn't exist yet, so it was just moved to a "Non-essential Built-in Functions" section in the docs. But it was retroactively considered to be deprecated since 2.3, as you can see in the 2.7 docs.

    3. Guido originally wanted to get rid of even them; the argument was that list comprehensions can do the same job better, as you can see in the "Regrets" flipbook. But promoting itertools.imap in place of map means it could be made lazy, like the new zip, and therefore better than comprehensions. I'm not sure why Guido didn't just make the same argument with generator expressions.

    4. I'm not sure Guido himself was ever convinced for reduce, but the core devs as a whole were.