Search code examples
pythonmultithreadingconcurrent.futures

Implementing multithreading on a list of objects


def return_total():
    dE_total = 0
    for num in range(len(self.layer)):
        dE_total += self.layer[num].backprop(delta[num])
    return dE_total

I have the above method inside a class. I need to call the backprop() method using multithreading. Usually the length of self.layer is small. I was planning to try ThreadPoolExecutor's map() method. As far as i know it is used to call a function and iterable value. Here each thread should execute for a different method with a input paramater. Is there any way to go about doing this?

with ThreadPoolExecutor() as executor:
    dE_total += executor.map(self.layer.backprop, delt)

I am aware the above code does not make any sense. I'm looking for something similar to the above idea.

Thanks for any help in advance


Solution

  • If I'm interpreting this correctly, you could write a method which takes the function as argument. This can then be passed to executor.map, e.g.:

    def func_caller(funcs, params):
        return func(*params)
    
    dE_total += sum(executor.map(func_caller, funcs_params))
    

    or similar, with funcs_params some appropriate list of tuples of functions and parameters. The argument unpacking might need to be adjusted.