Search code examples
pythonpython-3.xpython-decorators

mechanism behind decoration in python


Below is an example of decorator in python. I don't quite get how it actually works for the doubly decorated decorator.

from functools import update_wrapper
def decorator(d):
    print(d.__name__)
    return lambda fn: update_wrapper(d(fn),fn)

decorator=decorator(decorator) #I don't understand how this works.

@decorator
def n_ary(f):
    print(f.__name__)
    def n_ary_f(x,*args):               
        return x if not args else f(x,n_ary_f(*args))
    return n_ary_f

@n_ary
def seq(x,y):return ('seq',x,y)

It seems that the flow should be (I am not sure about it):

  1. decorator is decorated, so it returns lambda fn: update_wrapper(decorator(fn),fn).

  2. n_ary=decorator(n_ary), then n_ary is now updated due to the function of update_wrapper(decorator(n_ary),n_ary)

  3. The third part should be the update of seq, but I don't understand when is the update_wrapper function used.


Solution

  • Decoration is just syntactic sugar for calling another function, and replacing the current function object with the result. The decorator dance you are trying to understand is over-using that fact. Even though it tries to make it easier to produce decorators, I find it doesn't actually add anything and is only creating confusion by not following standard practice.

    To understand what is going on, you can substitute the function calls (including decorators being applied) with their return values, and tracking the d references by imagining saved references to the original decorated function object:

    1. decorator=decorator(decorator) replaces the original decorator function with a call to itself. We'll just ignore the print() call here to make substitution easier.

      The decorator(decorator) call returns lambda fn: update_wrapper(d(fn),fn), where d is bound to the original decorator, so now we have

      _saved_reference_to_decorator = decorator
      decorator = lambda fn: update_wrapper(_saved_reference_to_decorator(fn), fn)
      

      so update_wrapper() is not actually called yet. It'll only be called when this new decorator lambda is called.

    2. @decorator then calls the above lambda (the one calling _saved_reference_to_decorator(fr) and passing the result to update_wrapper()) and applies that lambda to the def n_ary(f) function:

      n_ary = decorator(n_ary)
      

      which expands to:

      n_ary = update_wrapper(_saved_reference_to_decorator(n_ary), n_ary)
      

      which is:

      _saved_reference_to_n_ary = n_ary
      n_ary = update_wrapper(lambda fn: update_wrapper(_saved_reference_to_n_ary(fn), fn), n_ary)
      

      Now, update_wrapper() just copies metadata from the second argument to the first returning the first argument, so that then leaves:

      n_ary = lambda fn: update_wrapper(_saved_reference_to_n_ary(fn), fn)
      

      with the right __name__ and such set on the lambda function object.

    3. @n_ary is again a decorator being applied, this time to def seq(x, y), so we get:

      seq = n_ary(seq)
      

      which can be expanded to:

      seq = update_wrapper(_saved_reference_to_n_ary(seq), seq)
      

      which if we take the return value of update_wrapper() is

      seq = _saved_reference_to_n_ary(seq)
      

      with the metadata copied over from the original seq to whatever the original n_ary function returns.

    So in the end, all this dance gets you is update_wrapper() being applied to the return value from a decorator, which is the contained wrapper function.

    This is all way, way too complicated. The update_wrapper() function has a far more readable helper decorator already provided: @functools.wraps(). Your piece of code could be rewritten to:

    import functools
    
    def n_ary(f):
        print(f.__name__)
        @functools.wraps(f)
        def n_ary_f(x,*args):
            return x if not args else f(x,n_ary_f(*args))
        return n_ary_f
    
    @n_ary
    def seq(x,y):return ('seq',x,y)
    

    I simply replaced the @decorator decorator on the n_ary() function definition with a @functools.wraps() decorator on the contained wrapper function that is returned.