Can you detect how well a client is keeping up with a stream of websocket messages?

I'm writing a video streaming service, and was thinking of streaming video via websockets.

A problem I foresee is that the client has insufficient bandwidth to receive the stream, so I want to be able to detect if I'm getting too far ahead of my client, and throttle down the messages to either a lower framerate or quality.

Can you detect when tornado is sending too much for the client to receive?

Solution

You don't have to worry about a slow network; but you do have to worry about a fast network.

You won't be able to write more data to the network than the client is able to accept. So you will not get ahead.

Let's say you're reading and sending the video in chunks. This is what your code may look like:

while True:
    self.write(chunk)
    await self.flush() # write chunk to network

The await self.flush() statement will pause the loop until the chunk has been written to the network. So if it's a slow network, it will pause for longer. As you can see you don't have to worry about getting far ahead of the client.

However, if your client's network is fast, then the flush operation will also be very fast and this may block your server because this loop will keep running until all the data has been sent and the IOLoop won't get a chance to serve other clients.

For those cases, Ben Darnell, tornado's maintainer, offered a very clever solution in a google forums thread which he calls:

serve each client at a "fair" rate instead of allowing a single client to consume as much bandwidth as you can give it.

Here's the code (taken directly from Ben Darnell's post):

while True:
    # Start the clock to ensure a steady maximum rate
    deadline = IOLoop.current().time() + 0.1

    # Read a 1MB chunk
    self.write(chunk)

    await self.flush()

    # This sleep will be instant if the deadline has already passed;
    # otherwise we'll sleep long enough to keep the transfer
    # rate around 10MB/sec (adjust the numbers above as needed
    # for your desired transfer rate)

    await gen.sleep(deadline)

Now, even if the flush operation is fast, in the next statement the loop will sleep until the deadline thereby allowing the server to serve other clients.