Search code examples
performanceserverarchitecturesystem-design

Handle 100K request per second


How can i design a system that handle 100k req/s at peak time, each request performs an update of an object within 20kb size?

I used nodejs as backend for non blocking io, write-behind pattern with redis as cache and mongodb as db, but it handles only 10k req/s at peak.


Solution

  • if your hardware resources are enough but you want to optimize your application you may consider some of the techniques bellow

    Cluster Module : use the cluster module of nodejs to get some advantages of multi core system this will allows you to create multiple instance of your application each of them is running on a CPU core

    Worker : for the expensive operations task(CPU bound) consider to use the worker threads this can separate task to multiple thread which able the main event loop responsible for other async task

    Job managers : You can using job managers like bull to do some operations in the background by adding them to a queue

    compression : you can using compression technique to reduce the data size use a good compress algorithm like Gzip or Brotli