Tornado's AsyncHTTPClient subclass

Question

(python 3.6.4 and tornado 4.5.3)

When using : http_client = tornado.httpclient.AsyncHTTPClient() the asynchronous http request fetching works fine.

But trying to define and use a subclass of AsyncHTTPClient got me into some kind of a deadlock when running the program (as opposed to the synchronous HTTPClient class where a subclass of it functioned well)

*Please correct me: If Tornado's AsynchHTTPClient class adheres to / inherits from the Configurable interface / abstract class , Then how it is possible to build objects from it? (Same rules as in Java?) . Is it that somehow something picks a default implementation for it among several built in ones?

"Configurable subclasses must define the class methods configurable_base and configurable_default, and use the instance method initialize instead of init. " - Will The default ctor in the case of inheritance will call super.init ? Is this the reason for the problem?

From the Documentation it appears that inheriting from AsyncHTTPClient is not recommended / not a valid method to use it:

    http_client = AsyncHTTPClient()
    http_client.fetch("http://www.google.com/", handle_response)

The constructor for this class is magic in several respects: It actually creates an instance of an implementation-specific subclass, and instances are reused as a kind of pseudo-singleton (one per .IOLoop). The keyword argument force_instance=True can be used to suppress this singleton behavior. Unless force_instance=True is used, no arguments should be passed to the AsyncHTTPClient constructor. The implementation subclass as well as arguments to its constructor can be set with the static method configure() All AsyncHTTPClient implementations support a defaults keyword argument, which can be used to set default values for HTTPRequest attributes. For example::

AsyncHTTPClient.configure(
    None, defaults=dict(user_agent="MyUserAgent"))
# or with force_instance:
client = AsyncHTTPClient(force_instance=True,
    defaults=dict(user_agent="MyUserAgent"))

additional questions:

1)Does buffering the response is a matter of choice?

3)When should I use the class tornado.web.RequestHandler ?

---

No errors at the moment but I'm not receiving a response after the actual fetch.

import sys
from tornado import ioloop, gen, httpclient

Under class SimpleAsyncHTTPClient(httpclient.AsyncHTTPClient):

#had to add this one (abstract function empty implementation? )
# I think that's the troublemaker
def fetch_impl(self, request, callback):
    pass

@gen.coroutine
def get(self, url):

    method = 'GET'
    print('send Async GET request ')
    res = yield self._fetch(url, method)
    print('after _fetch ...')
    return res

@gen.coroutine
def _fetch(self, url, method):
    print('send Asynchronous request ...')
    res = yield self.fetch(url, method=method)
    print('got a response')
    return res

Under a global:

@gen.coroutine

def ioloop_task():

yield gen.sleep(3)
url = 'http://google.com'
http_client = SimpleAsyncHTTPClient()

res = yield http_client.get(url)
res_code =  res.code
res_body =  res.body
print('return code: {}, body: {}....'.format(res_code, res_body[:60]))

print('do other stuff ....')
yield gen.sleep(2)

print(' task completed')

Answer 1

Since you've updated your question with additional details, and it has become very different from the original, I'd add another answer.

The magic behind AsyncHTTPClient

Consider this code:

http_client = AsyncHTTPClient()
print(http_client.__class__)

# Output:
<class 'tornado.simple_httpclient.SimpleAsyncHTTPClient'>

As you can see, http_client is an instance of SimpleAsyncHTTPClient, not AsyncHTTPClient. So, what's going on here?

If you look at the source code of AsyncHTTPClient, you'll see that it is inheriting the tornado.utils.Configurable class.

The most important piece of code in Configurable class is the __new__ method which is responsible for all the magic. If you look at its source code, you'll find that it will create an instance of whatever class is returned by the classmethod configurable_default.

Now, look at the source code of AsyncHTTPClient.configurable_default. It is returning the SimpleAsyncHTTPClient class, and that is why the object we created above (http_client), is the instance of SimpleAsyncHTTPClient, not of AsyncHTTPClient.

Finally, yes, you're right that you need to create the fetch_impl method in your subclass. Because AsyncHTTPClient will call the self.fetch_impl method. You can see this in this line in source code.

Though, fetch_impl hasn't been implemented in AsyncHTTPClient class, but it has been implemented in SimpleAsyncHTTPClient. You can find it here.

---

How to successfully subclass AsyncHTTPClient?

I'd start by a looking at the source code of SimpleAsyncHTTPClient and modify it to suit your needs.