Search code examples
pythonmultiprocess

Multiprocessing / Process - How to manage the queue/manager?


Currently what I execute the job, then 3 processes will be started on the same time.

def someFunctionJob(x,y,**kwargs):
   ..... Code which will do some job

def manager(x,values):
    y = uuid.uuid4().hex
    pr = Process(target=someFunctionJob,args=(x,y,),kwargs=values)
    pr.start()

def main(x,values,settings):

    for i in settings:
        if xyz in ["abc"]:
            manager(x,values)
        elif lender in ["abc1"]:
           manager(lender,product)
        elif lender in ["abc2"]:
            manager(x,values)

What I need is that it will be managed by queue:

E.G. if workers = 2

The execution will be like this:

process 1
process 2
waiting till any of the processes will be finished
process 3

Solution

  •   processes=[]
      for _ in range(2):
          p = Process(target=something )
          p.start()
          processes.append(p)
    
      for process in processes:
          process.join()
    
    p = Process(target=something )
    p.start()
    

    What this code does is it executes 2 processes above then waits them finish and run another separate process.