I have wrote a sample code of my problem. I am generating a random string and a shuffle function which adds a delay to a message so it comes out in a different order.
However, the scheduled task only executes once I have joinall at the end. Is there a way to execute the scheduling and tasks while dynamically scheduling new spawning ones. When I keep pressing enter, it schedules a new task but it doesn't execute until I have reached the random condition I have set. But, if I put the join/joinall after the append, it will block. Is this possible to do with gevent or what else libraries can this be done with any other asynchronous I/O or non-blocking libraries or do I have to resort to multithreading.
#!/usr/bin/python
import random
import string
from gevent import sleep, spawn, joinall
def random_string():
digits = "".join( [random.choice(string.digits) for i in xrange(8)] )
chars = "".join( [random.choice(string.letters) for i in xrange(10)] )
return chars
def delay_message(message, delay):
sleep(delay)
print("Shuffled message: {} and time: {}". format(message, delay))
def main():
while True:
s = raw_input("Please continue pressing enter, messages will appear when they are ready")
if s == "":
delay = random.randint(0, 10)
string = random_string()
print("Message: {} and time: {}". format(string, delay))
tasks = []
tasks.append(spawn(delay_message, string, delay))
if (random.randint(0,10) == 5): # random condition in breaking
joinall(tasks, raise_error=True)
break
else:
print("Exiting")
break
if __name__ == "__main__":
main()
I found a solution: Use ThreadPool e.g. tasks = ThreadPool(25) that is assigned outside the loop and remove tasks=[] and the joinall()
Also, add from gevent import wait then add wait() in the random condition in the while loop before the break, so that existing tasks can finish before the program is exited