Iterating a loop using await or yield causes error

Question

I come from the land of Twisted/Klein. I come in peace and to ask for Tornado help. I'm investigating Tornado and how its take on async differs from Twisted. Twisted has something similar to gen.coroutine which is defer.inlineCallbacks and I'm able to write async code like this:

kleinsample.py

@app.route('/endpoint/<int:n>')
@defer.inlineCallbacks
def myRoute(request, n):
    jsonlist = []
    for i in range(n):
        yield jsonlist.append({'id': i})
    return json.dumps(jsonlist)

curl cmd:

curl localhost:9000/json/2000

This endpoint will create a JSON string with n number of elements. n can be small or very big. I'm able to break it up in Twisted such that the event loop won't block using yield. Now here's how I tried to convert this into Tornado:

tornadosample.py

async def get(self, n):
    jsonlist = []
    for i in range(n):
        await gen.Task(jsonlist.append, {'id': i})    # exception here
    self.write(json.dumps(jsonlist))

The traceback:

 TypeError: append() takes no keyword arguments

I'm confused about what I'm supposed to do to properly iterate each element in the loop so that the event loop doesn't get blocked. Does anyone know the "Tornado" way of doing this?

Answer 1

You cannot and must not await append, since it isn't a coroutine and doesn't return a Future. If you want to occasionally yield to allow other coroutines to proceed using Tornado's event loop, await gen.moment.

from tornado import gen

async def get(self, n):
    jsonlist = []
    for i in range(n):
        jsonlist.append({'id': i})
        if not i % 1000:  # Yield control for a moment every 1k ops
           await gen.moment
    return json.dumps(jsonlist)

That said, unless this function is extremely CPU-intensive and requires hundreds of milliseconds or more to complete, you're probably better off just doing all your computation at once instead of taking multiple trips through the event loop before your function returns.