Search code examples
architecturedistributed-systemserver-pushlow-latency

Low latency, server push. How many open connections can server have?


The task is to send notifications to clients about some Events. Let's say for an example Event rate is about ~1 per second, we have about 10000 clients listening to the event. What architecture would you suggest? How many open connections are possible to maintain for server push?


Solution

  • If you use a websocket server based on a good scalable multithreaded architecture, the real limit is not the number of open connections but the total throughput. In your case, the total throughout is 10,000 messages per second. We should also take into account the average message size. For example if a typical message is 10 bytes, your total throughput is 0.8 Mbit/s; if a typical message is 1000 bytes, your total throughput is 80 Mbit/s.

    In both cases, in our experience a mid-sized server can safely handle such traffic.

    Just for reference, we successfully tested 1 million concurrent connections on our websocket server with a per-connection throughput of 1 message every 10 seconds. That means 100,000 messages per second spread over 1,000,000 sockets on a single server. [Full disclosure: I am the co-founder of Lightstreamer]