IPython.parallel client is hanging while waiting for result of map_async

Question

I am running 7 worker processes on a single machine with 4 cores. I may have made a poor choice with this loop while waiting for the result of map_async:

while not result.ready():
    time.sleep(10)
    for out in result.stdout:
        print out
rec_file_list = result.get()

result.stdout keeps growing with all the printed output from the 7 processes running, and it caused the console that initiated the map to hang. The activity monitor on my MacBook Pro shows the 7 processes are still running, and the terminal running the Controller is still active. What are my options here? Is there any way to acquire the result once the processes have completed?

Answer 1

I found an answer: Remote introspection of ASyncResult objects is possible from another client as long as a 'database backend' has been enabled by the controller with:

ipcontroller --dictb # or --mongodb or --sqlitedb

Then, it is possible to create a new client instance and retrieve the results with:

client.get_result(task_id)

where the task_ids can be retrieved with:

client.hub_history()

Also, a simple way to avoid the buffer overflow I encountered is to periodically print just the last few lines from each engine's stdout history, and to flush the buffer like:

from IPython.display import clear_output
import sys

while not result.ready():
    clear_output()
    for stdout in result.stdout:
        if stdout:
            lines = stdout.split('\n')
            for line in lines[-4:-1]:
                if line:
                    print line
    sys.stdout.flush()
    time.sleep(30)