Search code examples
pythonmultithreadingcopypool

python thread pool copy parameters


I'm learning about multithreading and I try to implement a few things to understand it. After reading several (and very technical topics) I cannot find a solution or way to understand my issue.

Basically, I have the following structure:

class MyObject():
    def __init__():
        self.lastupdate = datetime.datetime.now()
    def DoThings():
        ...

def MyThreadFunction(OneOfMyObject):
    OneOfMyObject.DoThings()
    OneOfMyObject.lastupdate = datetime.datetime.now()

def main():
    MyObject1 = MyObject()
    MyObject2 = MyObject()
    MyObjects = [MyObject1, MyObject2]

    pool = Pool(2)

    while True:
        pool.map(MyThreadFunction, MyObjects)

if __name__ == '__main__':
    main()

I think the function .map make a copy of my objects because it does not update the time. Is it right ? if yes, how could I input a Global version of my objects. If not, would you have any idea why the time is fixed in my objects ?

When I check the new time with print(MyObject.lastupdate), the time is right, but not in the next loop

Thank you very much for any of your ideas


Solution

  • Yes, python threading will serialize (actually, pickle) your objects and then reconstruct them in the thread. However, it also sends them back. To recover them, see the commented additions to the code below:

    class MyObject():
        def __init__():
            self.lastupdate = datetime.datetime.now()
        def DoThings():
            ...
    
    def MyThreadFunction(OneOfMyObject):
        OneOfMyObject.DoThings()
        OneOfMyObject.lastupdate = datetime.datetime.now()
    
        # NOW, RETURN THE OBJECT
        return oneOfMyObject
    
    def main():
        MyObject1 = MyObject()
        MyObject2 = MyObject()
        MyObjects = [MyObject1, MyObject2]
    
        with Pool(2) as pool: # <- this is just a neater way of doing it than a while loop for various reasons. Checkout context managers if interested.
    
            # Now we recover a list of the updated objects:
            processed_object_list = pool.map(MyThreadFunction, MyObjects)
    
        # Now inspect
        for my_object in processed_object_list:
                print(my_object.lastupdate)
    
    
    if __name__ == '__main__':
        main()