Search code examples
pythonpartialtimeitfunctools

Timeit and Partials


Check this python code:

 from functools import partial
 def summer(x, y):
      return x+y

 >>>partial(summer, 1, 2)
 <functools.partial object at 0x10e648628>

However, when I pass in partial time timeit:

  import timeit
  min(timeit.repeat(partial(summer, 1, 2)))

It seems to evaluate summer with x=1 and y=2.

Am I understanding what's going on? Am I using timeit right? And how do I get partial to evaluate summer, when I pass values for every parameter?

EDIT: I added the below comment as part of this question for clarity.

My issue is that partial(summer, 1, 2) does not evaluate summer. So why does calling it inside of timeit.repeat evaluate summer?


Solution

  • In Python function objects are "first-class" objects. They can be passed as arguments to functions just like any other object. In this case, timeit.repeat is being passed the function object partial(summer, 1, 2) so that later, inside of repeat the function object can be called and timed.

    partial(summer, 1, 2) itself is another example of passing a function (summer) as an argument to another function (partial). It makes sense that you'd want to pass the function object since you don't want to call summer yet. Instead you want the function object returned by partial to have access to summer so it can be called later when that function object is called.


    partial(summer, 1, 2) returns a function object:

    In [36]: partial(summer, 1, 2)
    Out[36]: <functools.partial at 0x7ff31f0bd3c0>
    

    As you know, to call the function, you need to place parentheses after the function:

    In [37]: partial(summer, 1, 2)()
    Out[37]: 3
    

    Normally when I use timeit I pass a statement stmt and setup to the timeit function as strings:

    In [41]: func = partial(summer, 1, 2)
    
    In [42]: timeit.repeat('func()', 'from __main__ import func')
    Out[42]: [0.11481308937072754, 0.10448503494262695, 0.1048579216003418]
    

    However, you are correct that it is also possible to pass a callable (such as a function object) as the first argument:

    timeit.repeat(func) 
    

    repeat will call func and time the result. You can see how repeat handles this case by stepping through the code using a debugger like pdb:

    import pdb
    pdb.set_trace()
    timeit.repeat(func)
    

    Inside the code for timeit.py around line 140 you'll see:

        elif callable(stmt):
            self.src = None
            if isinstance(setup, str):
                _setup = setup
                def setup():
                    exec(_setup, global_ns, local_ns)
    

    which checks if stmt, the first argument is callable. If it is, then it sets func to stmt, and later calls func (source code).

    def inner(_it, _timer, _func=func):
        setup()
        _t0 = _timer()
        for _i in _it:
            _func()       # <--- The function call happens here
        _t1 = _timer()
        return _t1 - _t0
    return inner