Search code examples
pythonasynchronoustornadopycurl

Tornado's AsyncHTTPClient no longer works after upgrade to 2.0 from 1.2


Decided I'd kick the tires of Tornado 2.0 tonight, but it appears to have done a number on ASyncHTTPClient for me. Nothing in the release notes for 2.0 indicates any real changes are necessary to how I'm using ASyncHTTPClient:

[EDIT: made the code more of an explicit, self-containing example ]

import time
import threading
import functools

import tornado.ioloop
import tornado.web
from tornado.httpclient import *

class MainHandler(tornado.web.RequestHandler):

    def perform_task(self,finish_function):
      http_client = AsyncHTTPClient()
      tornado.ioloop.IOLoop.instance().add_callback(finish_function)
      # do something
      for i in range(0,10):
        print i
        time.sleep(1)
      request = tornado.httpclient.HTTPRequest("http://10.0.1.5:8888",method="POST",body="finished countdown")
      resp = http_client.fetch(request, self.handle_request)
      return

    def join_callback(self):
      # finish the request, also returns control back to ioloop's thread.
      self.finish()

    def handle_request(self, response):
      if response.error:
          print "Error:", response.error
      else:
          print response.body

    @tornado.web.asynchronous
    def get(self):
        self.write("Kicking off.")
        a_partial = functools.partial(self.perform_task,self.join_callback)
        self.thread = threading.Thread(target=a_partial)
        self.thread.start()
        self.write("\n<br/>Done in here, out of my hands now.")

    # just so this example has something to post to
    @tornado.web.asynchronous
    def post(self):
        self.write("POSTED: %s" % (self.request.body))

application = tornado.web.Application([
    (r"/", MainHandler),
])

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

when using the default (non-curl) ASyncHTTPClient I get the following:

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/tornado/simple_httpclient.py", line 259, in cleanup
    yield
  File "/usr/local/lib/python2.6/dist-packages/tornado/simple_httpclient.py", line 186, in __init__
    functools.partial(self._on_connect, parsed))
  File "/usr/local/lib/python2.6/dist-packages/tornado/iostream.py", line 120, in connect
    self.socket.connect(address)
AttributeError: 'NoneType' object has no attribute 'connect'

if I add in: tornado.httpclient.AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")

to specify I want to use PyCurl, I get the following exception:

   Traceback (most recent call last):
      File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
        self.run()
      File "/usr/lib/python2.6/threading.py", line 484, in run
        self.__target(*self.__args, **self.__kwargs)
      File "migratotron.py", line 46, in perform_migration
        hc.fetch(request,success)
      File "/usr/local/lib/python2.6/dist-packages/tornado/curl_httpclient.py", line 81, in fetch
        self._process_queue()
      File "/usr/local/lib/python2.6/dist-packages/tornado/curl_httpclient.py", line 210, in _process_queue
        curl.info["headers"])
      File "/usr/local/lib/python2.6/dist-packages/tornado/curl_httpclient.py", line 276, in _curl_setup_request
        curl.setopt(pycurl.URL, request.url

)
TypeError: invalid arguments to setopt

The only thing unusual I'm doing is calling this is another thread I created to do some background processing, which uses add_callback to return control to the ioloop thread. Has anyone seen this with 2.0 or anything like it?


Solution

  • Those error messages are a bit cryptic, but in both cases I think they're due to your request not having a URL (unless callback is a URL).

    2.0 switched the ASyncHTTPClient implementation to simple_httpclient (from curl_httpclient), but I don't think the example given would have worked in Tornado 1.2 either.