Tornado doesnt yield futures correctly

Question

I need to asynchronously make several http calls inside a tornado request handler.

trying to return futures is not documented well and pretty much impossible to do on a gathered asyncio.gather at the handler level of tornado.

I have tried aiohttp which works fine by itself however when putting it inside a tornado handler it throws loop is already in use. If you can show me how to inject into the the IOLoop some new futures to resolve that would be nice.

I have also tried using tornados AsyncHTTPClient which contrary to the documentation does not actually use yield but returns the response when you use await on it.

Is there any up to date documentation on this? ALL the example do not work for multiple async requests.

according to this documentation http://www.tornadoweb.org/en/stable/gen.html#module-tornado.gen

@gen.coroutine
def get(self):
    http_client = AsyncHTTPClient()
     response1, response2 = yield [http_client.fetch(url1),
                              http_client.fetch(url2)]
    response_dict = yield dict(response3=http_client.fetch(url3),
                           response4=http_client.fetch(url4))
    response3 = response_dict['response3']
    response4 = response_dict['response4']

however when trying to do this myself yield throws an error, substituting that with await gets a result. however you cannot await on a dict object like yield can. How can i get around this?

python 3.6.7 tornado 5.1.1 aiohttp 3.5.4

Answer 1

Your comments use the word await, so it sounds like you're running in to a difference between native coroutines (defined with async def and await) and decorated coroutines (defined with @gen.coroutine and yield). You can only yield lists and dictionaries directly in decorated coroutines.

In a native coroutine, you must use tornado.gen.multi (or asyncio.gather):

async def get(self):
    http_client = AsyncHTTPClient()
    response1, response2 = await gen.multi([http_client.fetch(url1),
                                            http_client.fetch(url2)])
    response_dict = await gen.multi(dict(response3=http_client.fetch(url3),
                                         response4=http_client.fetch(url4)))
    response3 = response_dict['response3']
    response4 = response_dict['response4']

The differences between the two styles of coroutines are documented here