Search code examples
pythonpython-multithreading

Python multithreading with Classes that return values


So I know how basic threading in Python works, but here is my issue:

  1. I have two custom python modules that each return a value (one returns a dictionary and the other datetime object)
  2. I am importing them to a third py script file (using the import function), which make use of these returned values
  3. The two modules are not dependent on each other.
  4. The one that returns the dictionary takes twice as much time as the other one (which is the reason this issue arose).
  5. Both the custom modules are bound within a Class, with __init__, so it's like this:
Class A:
    def __init__(self, var1, var2):
        // the regular self.var assignments
        
    def Aa(self):
        // process the input values
        return dict_value

and

Class B:
    def __init__(self, var3):
        // the regular self.var assignments
        
    def Bb(self):
        // process the input values
        return datetime_object

Why am I writing all this? Read on:

Naturally, since once function takes longer than the other, I'd like to optimize the program. I am trying to implement threading, but since my modules are imported externally PLUS the fact that they return values, I am finding it a little challenging. The closest I've been is this answer: https://stackoverflow.com/a/22177060/1725974 - basically this part:

from threading import Thread
from Queue import Queue

def func1(x):
    return [i*i for i in x]

def func2(x):
    return [i*i*i for i in x]

nums = [1,2,3,4,5]

def wrapper(func, arg, queue):
    queue.put(func(arg))

q1, q2 = Queue(), Queue()
Thread(target=wrapper, args=(func1, nums, q1)).start() 
Thread(target=wrapper, args=(func2, nums, q2)).start() 

print q1.get(), q2.get()

This is all OK, but I am not able to figure out how I can pass the arguments in the wrapper function. Since I use init in my class, I pass the arguments in the class itself, then call the definition inside it. I can't seem to figure out how I can do that.

For example, here's what I am doing and it isn't working (obviously): Thread(target=wrapper, args=(A(var1, var2).Aa, q1)).start()

The error reads "TypeError: wrapper() missing 1 required positional argument: 'queue'"

Even if I declare the arguments early on (like how nums is declared), how will args know where to pass the value to? I am not getting this.

Thanks in advance.


Solution

  • It looks like you need a different wrapper function to call the methods Aa and Bb since they have no arguments as such:

    def wrapper(func, queue):
        queue.put(func())
    
    q1, q2 = Queue(), Queue()
    Thread(target=wrapper, args=(A(var1, var2).Aa, q1)).start() 
    Thread(target=wrapper, args=(B(var3).Bb, q2)).start() 
    
    print q1.get(), q2.get()