I'm intending to receive a binary stream of data via a http POST call.
I believe the client side is working, that is, it writes chunks of bytes to the server, I can see the amount of data being sent with tcpdump, yet Twisted's request.content file-like object only starts producing output once the client disconnects.
This is what the server handler looks like:
def render(self, request):
if request.path == '/incoming-stream':
d = deferLater(reactor, 0, lambda: request)
d.addCallback(self.async_read)
return NOT_DONE_YET
def async_read(self, request):
sys.stdout.write('\nasync_read ' + str(request) + '\n')
sys.stdout.flush()
while True:
byte = request.content.read(1) # <--- read one byte
if len(byte) > 0:
sys.stdout.write(repr(byte))
sys.stdout.flush()
else:
break
sys.stdout.write('\nfinished ' + str(request) + '\n')
sys.stdout.flush()
request.write(b"finished")
request.finish()
If I can't do this with POST, I have no problem with switching over to WebSocket, but I'd first like to try to get this done via POST. The data posted are long running (one new POST request every hour, with it being alive and receiving data for an hour), relatively high bandwidth sensor data at approx 1kbps.
I am aware that there are better methods of transferring the data (WebSocket, MQTT, AMQP), but POST and WebSocket will give me least amount of trouble when receiving the data through an NGINX SSL endpoint. Currently NGINX is not being used (to discard any buffering it could be causing).
Twisted Web does not support streaming uploads in its IResource abstraction.