Search code examples
pythonpython-3.xpython-multithreading

A clean API for making a function call threaded in python


I would like to call a function in a thread. Calling it with the conventional API looks like:

from threading import Thread
import numpy as np
a = np.random.rand(int(1e8),1)

Thread(target=np.savez_compressed, args=('/tmp/values.a', dict(a=a))).start()

I was wondering if there is a pythonic was of making this threaded call with a cleaner API, without defining a function which is specific for np.savez_compressed.

E.g. something in the style of (pseudo-code):

@make_threaded
np.savez_compressed('/tmp/values.a', dict(a=a))

Unfortunately decorators can only be applied to function definitions, so the pseudo-code above is not legal.

EDIT: I am not looking specifically for a decorator API. Rather, a cleaner way to make a function call threaded


Solution

  • The concurrent.futures module provides a more high-level API for using threads or processes for individual operations.

    from concurrent.futures import ThreadPoolExecutor
    
    executor = ThreadPoolExecutor()
    executor.submit(np.savez_compressed, '/tmp/values.a', dict(a=a))
    

    If you don't want the entire Executor API, you can define your own helper to run a function in a thread.

    def threaded(call, *args, **kwargs):
        """Execute ``call(*args, **kwargs)`` in a thread"""
        thread = threading.Thread(target=call, args=args, kwargs=kwargs)
        thread.start()
        return thread
    
    threaded(np.savez_compressed, '/tmp/values.a', dict(a=a))