Search code examples
pythonfork

Python: Share variable between forked child and parent


I am quite beginner to Python and I have a problem for sharing variable between parent script and forked child.

The objectif is as follows

The main script has to get tasks from an API call and run them in a subprocess. Then it decrement a counter of availables CPU to avoid ressource saturation and gets a new tasks if there are availables. I would like also to increment back the counter when the subprocess is finished but I can't get it works.

Here is my script main fonction

def main():
    while True:
        global availables_cpu

        print("Got %s availables CPU requesting new job" % availables_cpu)
        response = requests.get("http://" + url_api + "/api.php?action=get&ressources=" + str(availables_cpu))
        data = json.loads(response.text)
        job_path = data["hash"]

        if job_path is None :
            print("Nothing to do, waiting for a while")

        else:
            dir = job_path.rstrip()

        if int(data["cpu"]) <= availables_cpu:
            availables_cpu -= int(data["cpu"])
            newpid = os.fork()

        if newpid == 0:
            code = child()
            availables_cpu += int(data["cpu"])
            os._exit(code)
        else:

            time.sleep(3)
            print("Parent")

The child() function just gets the data and run the script in a subprocess

I'm not sure about the best way to do this and I'm opened in any suggestions.


Solution

  • As suggested by @polku multiprocessing.Value works well

    With these modifications

    availables_cpu = multiprocessing.Value('I',psutil.cpu_count())
    

    and replace all the occurences of availables_cpu by availables_cpu.value