Python generators and reduce

Question

I am working on a Python3 tornado web server with asynchronous coroutines for GET requests, using the @gen.coroutine decorator. I want to use this function from a library:

@gen.coroutine
def foo(x):
    yield do_something(x)

which is simple enough:

@gen.coroutine
def get(self):
    x = self.some_parameter
    yield response(foo(x))

Now assume there are multiple functions foo1, foo2, etc. of the same type. I want to do something like ...foo3(foo2(foo1(x).result()).result())... and yield that instead of just response(foo(x)) in the get method.

I thought this would be easy with reduce and the result method. However, because of how tornado works, I cannot force the foos to return something with the result method. This means that yield reduce(...) gives an error: "DummyFuture does not support blocking for results". From other answers on SO and elsewhere, I know I will have to use IOLoop or something, which I didn't really understand, and...

...my question is, how can I avoid evaluating all the foos and yield that unevaluated chunk from the get method?

Edit: This is not a duplicate of this question because I want to: 1. nest a lot of functions and 2. try not to evaluate immediately.

Answer 1

In Tornado, you must yield a Future inside a coroutine in order to get a result. Review Tornado's coroutine guide.

You could write a reducer that is a coroutine. It runs each coroutine to get a Future, calls yield with the Future to get a result, then runs the next coroutine on that result:

from tornado.ioloop import IOLoop
from tornado import gen


@gen.coroutine
def f(x):
    # Just to prove we're really a coroutine.
    yield gen.sleep(1)
    return x * 2


@gen.coroutine
def g(x):
    return x + 1


@gen.coroutine
def h():
    return 10


@gen.coroutine
def coreduce(*funcs):
    # Start by calling last function in list.
    result = yield funcs[-1]()

    # Call remaining functions.
    for func in reversed(funcs[:-1]):
        result = yield func(result)

    return result


# Wrap in lambda to satisfy your requirement, to 
# NOT evaluate immediately.
latent_result = lambda: coreduce(f, g, h)
final_result = IOLoop.current().run_sync(latent_result)
print(final_result)