Tornado async http client blocks

theQueue = tornado.queues.Queue()
theQueue.put_nowait('http://www.baidu.com')
theQueue.put_nowait('http://www.google.com')
theQueue.put_nowait('http://cn.bing.com/')

@tornado.gen.coroutine
def Test1():
    def cb(response):
        print str(response)

    while True:
        item = yield theQueue.get()
        print item
        tmp = tornado.httpclient.AsyncHTTPClient(force_instance=True)
        tmp.fetch(item,callback=cb)

@tornado.gen.coroutine
def Test2():
    while True:
        item = yield theQueue.get()
        print item
        tmp = tornado.httpclient.AsyncHTTPClient(force_instance=True)
        response = yield tmp.fetch(item)
        print str(response)

#Test1()
Test2()
tornado.ioloop.IOLoop.instance().start()

python 2.6 and tornado 4.2
In the function Test1,it will first prints out 3 items,then prints 3 responses.
But in Test2,it will print item and it's response one by one.

I was confused,why in Test2 isn't asynchronous?

Solution

Test2() is asynchronous, but in a different way, the coroutine way.

Tornado's coroutine suspends when it meets yield keyword, waiting for the async process(in your case, requesting webpage via http client) to complete. tornado will switch to other available coroutines when current coroutine suspends.

With coroutine, your code looks synchronous, and "runs synchronous"(if there is only one coroutine) too.

You can easily test the ASYNC feature of tornado's coroutine, by using two or more conroutines:

@tornado.gen.coroutine
def Test2():
    while True:
        item = yield theQueue.get()
        print 'Test2:', item
        tmp = tornado.httpclient.AsyncHTTPClient(force_instance=True)
        response = yield tmp.fetch(item)
        print 'Test2:', str(response)

# Write another test function called `Test3` and do the exactly same thing with Test2.
@tornado.gen.coroutine
def Test3():
    while True:
        item = yield theQueue.get()
        print 'Test3:', item
        tmp = tornado.httpclient.AsyncHTTPClient(force_instance=True)
        response = yield tmp.fetch(item)
        print 'Test3:', str(response)

Test2()
Test3()
tornado.ioloop.IOLoop.instance().start()

You will see Test2 and Test3 runs simultaneously(but not really) in this example.

The ability of switching between different routines to perform concurrent operations, that's the meaning of coroutine asynchronous.