I am attempting to run an infinite while loop that will call a function that makes an API call. Each API call can take between 9-12 seconds and I want x processes constantly running.
I've attempted some of the existing pool examples but they seem to take a function along with a list of arguments. My function generates all inputs needed.
from multiprocessing import Process
from random import randint
from time import sleep
def loop_a():
while 1:
wait = randint(5,9)
print("now waiting" + str(wait) + " seconds")
sleep(wait)
if __name__ == '__main__':
#Initialize two separate while loops that can call/wait for responses independently
Process(target=loop_a).start()
Process(target=loop_a).start()
This sample code I found from another question solves my problem mostly, but I am wondering if there is an elegant way to define how many processes to run at once. Id like to be able to enter the number of processes as a parameter rather than defining a new line for each process. Is there a nice way to accomplish this?
This snippet seems to fix the issue I was facing.
if __name__ == '__main__':
[Process(target=loop_a).start() for x in range(2)]
You should use Pool
from multiprocessing import Pool
pool = Pool(processes=2) # define number of processes
results = [pool.apply(loop_a) for x in range(2)]