Search code examples
pythonpython-3.xpython-multiprocessing

Variable does not change in Python multiprocessing


I'm trying to use a dictionary in my multiprocessing code. The problem is when I changed the value of one key in the process, the changes just affect only that process. Apparently, variable arguments of the process are a copy of that variable. I tried to use dict method but it does not work too. What is my mistake?

import time 
from multiprocessing import Process, Manager     

def f1(list1, set1):
    list1.append(3)
    set1['func']='f1' 
    print('f1 dic',set1) 
    print('f1 list',list1) 
    while True:
        time.sleep(3)
        print('f1 dic',set1) 
        print('f1 list',list1) 

def f2(list1): 
    list1.append(4)
    settings_dic['func']='f2'
    print('f2 dic',settings_dic) 
    print('f2 list',list1) 
    while True:
        time.sleep(3)
        print('f2 dic',settings_dic) 
        print('f2 list',list1) 

if __name__ == '__main__':     
    # list1 = Manager().dict()    
    settings_dic = Manager().dict()  
    Global = Manager().Namespace()
    list1 = [1,2]
    settings_dic = { 
        'mode' : 'master', 
        'logger' : 'True',
        'func' : ''
    }       
    p_video_get = Process(target=f1, args=(list1,settings_dic,))
    p_video_get.daemon = True
    p_video_get.start() 
    p_packetTrasnmission = Process(target=f2, args=(list1,))
    p_packetTrasnmission.daemon = True
    p_packetTrasnmission.start()  
    
    list1.append(5)
    settings_dic['func'] = 'main'
    print('main dic',settings_dic)
    print('main list',list1)  

    while True:
        time.sleep(3)
        print('main dic',settings_dic)
        print('main list',list1)  


Solution

  • UPDATE:

    As pointed out by juanpa.arrivillaga, the problem with the code above is that the Manager.dict() is re-assigned as a normal dict meaning it's lost the reference to the Manager.dict() and isn't changing/passing that one around. The rest is my original response where I thought the OP was just trying to pass a dictionary around.

    ORIGINAL:

    When you start up a separate processes, that process has their own address space. Unless you set something up, they don't share any memory (and by extension, data). Each process gets a copy of the arguments you pass in, and their process's memory manage their own copy of it (this is done by pickling the object and sending it to the new process). Once it's copied, they do not sync up. To have one process change the data in another you have a couple of options.

    You either need to setup some IPC via a shared comms or shared object. A Queue is an easy way to post updates to a data structure that other processes can then pop off and update their own copies.

    Or if you are on 3.8+, you can setup a shared memory. I've not done this, and you'd still need to write the code to monitor and decode the changes. But it's another way to have one process communicate with another.