Unable to Inter-Lock between processes while using multiprocessing module in Python

Question

I'm somewhat new to Python. I've been learning and researching about the Multiprocessing module and I'm stuck at this issue while using locking.

Here is the test code:

import multiprocessing
from multiprocessing import Pool

class Param_Class():

#Initialisation.
    def __init__(self):
        self.counter = multiprocessing.Value('i',0)
        self.lock = multiprocessing.Lock()

    def inc_counter(self):
            lk.acquire()
            print('Locked {}'.format(multiprocessing.current_process().name))
            self.counter.value+=1
            lk.release()
            print('Released {}'.format(multiprocessing.current_process().name))

param_class = Param_Class()

def init(lock):
    global lk
    lk = lock


def target():
    param_class.inc_counter()
    code = 0
    return code

def handler(i):
    if i<50:
        code = target()
        print(param_class.counter.value)


if __name__ == '__main__':
    p = Pool(processes=2,initializer=init, initargs=(param_class.lock,))
    p.map_async(handler,range(50))
    p.close()
    p.join()

What I expected is each process getting locked while the other one is executing so my counter should increment every time either of the process is executed. However, there is no interlocking happening between the process and the second process is working on the copy of the counter.

Here is the output:

Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
1
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
2
Released SpawnPoolWorker-2
1
Locked SpawnPoolWorker-1
Locked SpawnPoolWorker-2
Released SpawnPoolWorker-1
Released SpawnPoolWorker-2
3
2
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
4
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
3
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
5
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
4
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
6
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
5
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
7
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
6
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
8
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
7
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
9
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
8
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
10
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
9
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
11
Released SpawnPoolWorker-2
10
Locked SpawnPoolWorker-1
Locked SpawnPoolWorker-2
Released SpawnPoolWorker-1
Released SpawnPoolWorker-2
12
11
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
13
Released SpawnPoolWorker-2
12
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
14
Released SpawnPoolWorker-2
13
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
15
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
14
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
16
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
15
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
17
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
16
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
18
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
17
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
19
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
18
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
20
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
19
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
21
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
20
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
22
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
21
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
23
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
22
Released SpawnPoolWorker-1
24
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
25
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
26
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
27
Locked SpawnPoolWorker-1
Released SpawnPoolWorker-1
28

Desired Output should be something like this:

Locked SpawnPoolWorker-1
1
Released SpawnPoolWorker-1
Locked SpawnPoolWorker-2
2
Released SpawnPoolWorker-2
Locked SpawnPoolWorker-1
3
Released SpawnPoolWorker-1
.
.
.
.
.
.
.
Locked SpawnPoolWorker-2
50
Released SpawnPoolWorker-2

I've referred to this to learn about initializing a lock and inheriting it. I'm not able to comprehend where I've messed up. Is it the lock or the multiprocessing.Pool call or using class instances or perhaps the target function itself. Please be detailed about it with some theory to back it up. I'd like to learn about this, not just solve it.

Notice how, in the obtained output, it works as expected from value 24 to 28 and process 2 just stopped working. Need clarification on that one too.

P.S.: I'd want to avoid using a Manager as it is heavyweight as mentioned in the linked answer. Also, I don't want to eliminate the partial concurrency of the execution because I'll be using other variables which don't need to be locked and I want the module to stay fast and pythonic.

Answer 1

Apparently, every time the process undergoes an apply, the init of class is called, hence creating a copy for each process.

Hence to overcome this, instead of creating counter as a class attribute, it can be passed in the pool's init and made global.

Also, the lock is insignificant.