Search code examples
pythontornadopyzmq

Creating a ZMQStream for using in tornado IOloop


I'm trying to add a callback to my tornado application when a ZMQ message comes through. I'm getting:

*** TypeError: argument must be an int, or have a fileno() method.

When trying to create the ZMQStream object, I've read this happens when zmq ioloop is not installed, but I'm calling ioloop.install() from zmq in the first line of my program. calling zmq.eventloop.ioloop.IOLoop.instance() and tornado IOLoop.instance() returns the same object.

The interesting code is:

launch.py

import logging
import os
import tornado
import zmq
from zmq.eventloop import ioloop
from tornado.httpserver import HTTPServer
from web.apiserver import Userform, UploadHandler, UserHandler, LoginHandler

application = tornado.web.Application([
    (r"/upload/(?P<token>\w+)", UploadHandler),
    # .....
], debug=True)

if __name__ == "__main__":
if os.geteuid() != 0:
    exit('Por favor, ejecute como root')
ioloop.install()
logging.basicConfig(filename=r'/tmp/tornado.log', level=logging.INFO)
httpsserver = HTTPServer(application, ssl_options={
    "certfile": "web/server.crt",
    "keyfile": "web/server.key",
})
httpsserver.listen(8888)
zmq.eventloop.IOLoop.instance().start()

server.py

thread_pool = ThreadPoolExecutor(max_workers=4)
context = zmq.Context()
class UploadHandler(RequestHandler):
    # .......
    # after some callback magic we end up here:
    def watch_job(self, ip, port):
        subscriber_socket = context.socket(zmq.SUB)
        subscriber_socket.connect("tcp://%s:%s" % (ip, port))
        subscriber_socket.setsockopt(zmq.SUBSCRIBE, 'client')
        import ipdb; ipdb.set_trace()
        stream_sub = ZMQStream(subscriber_socket, IOLoop.instance()) #<--- Here TypeError
        stream_sub.on_recv(self.check_job_progress)

Other things I've tried:

  • Calling ioloop.install() after the tornado ioloop has started
  • Creating the socket before tornado ioloop has started
  • Praying

Solution

  • The call to zmq.ioloop.install() needs to be even earlier. Creating an Application in debug mode will initialize the default IOLoop, so you must install the ZMQ IOLoop before constructing the Application.