Search code examples
pythonjobssubprocessparallel-processing

python parallel programming

hi i need some guidelines how to write programm that executes other python programm but in maximum for example 6 times at once and always try to be 6 process be running even if one ends up

also i would like to know what is happening to those processes right know but i dont want to wait to any process to finish

what is the pid of just created process? and its still running? or there has been an error? or it finish sucessfully?

some job manager ...

import subprocess

def start():
    proc = {}
    for i in range (0,6):
        proc[i] = subprocess.Popen(
            ['python', 'someprogramm.py', '--env', 'DEVELOPMENT', '-l'],
            shell = True, 
            stdout = subprocess.PIPE, 
            stderr = subprocess.STDOUT
        )


if __name__ == '__main__':
    start()
Solution

  • Use celery.