I've got Tornado setup to act as a Websocket server, and it works fine.
However, I'd like to take the incoming stream of data on the websocket, queue it up, and upload it to a second server via an HTTP POST request with chunked-transfer encoding in realtime. I'm using the python requests library to do the POST (it supports getting the chunks from a generator).
Without coroutines, this does not work - as the upload method blocks execution, so new incoming data doesn't get added to the queue until after the upload has finished.
I added coroutines through some decorators in my websocket methods, but it doesn't work at all now - nothing gets uploaded via the HTTP POST. I even tried adding a yield inside the make_request function like so (as "any function that calls a coroutine must be a coroutine itself, and use the yield keyword in the call" according to the Tornado docs):
data = yield self.read_queue()
but that threw a syntax error. Example code is here.
I'm on Python 3.4.
The requests library is synchronous. You should not use it from Tornado except by running it in a thread. Using requests in a thread in this case would be trickier than the typical use of ThreadPoolExecutor in Tornado because you would be feeding data into it while it is running and you could have problems if a thread pool were exhausted.
Instead, use Tornado's AsyncHTTPClient. The body_producer argument allows you to upload the chunks as they come in.