Behavior of NextJs app under heavy traffic

Question

I am developing NextJs application. It will be deployed on Azure where I will have nodejs installed and will run next start -p 4000 command.

What I would like to know is how does NextJs handle heavy traffic? Namely, if there are something like 20k users going through my site, is this something that Nextjs can handle out of the box or should I dockerize and orchestrate multiple nodejs docker images with multiple nextjs applications?

Or, is Nextjs serving static files to my CDN so that I do not have to care about traffic stress of my nodejs where nextjs server is running?

Hope my question makes sense.

Answer 1

No magic number

There is no set number for capacity limit that can be pulled out of a hat. Next, and Node.js apps in general, are pretty efficient at handling multiple connections, but how heavy your load is depends on your site. How many simultaneous connections you can "handle" also depends on how much latency you find acceptable. For example, your server may be able to handle 40k simultaneous requests with 1 second of latency, but only 5k simultaneous requests with 100ms of latency.

Factors affecting capacity

How much traffic your server can handle will depend on things like:

• Amount of IO your server does. This includes data being sent to browsers, as well as data being read from disk or from a database. If you have a lot of static content (e.g. large images, videos) being served, this will probably start to limit you.
• Amount of processing your server does. This is how much code needs to run every API call. Usually this is pretty low and most servers are IO-bound, but sometimes there is a lot of processing (e.g. retrieving large data set from database and transforming it).
• Processing capacity of the machine upon which your server runs. All of your processing will be slower on a slower machine (fewer gigahertz means slower), so the processing that you do (described above) will take longer to run, which means you will block new connections for longer, which will lower the capacity of your server.
• IO speed of the machine upon which your server runs. This includes disk speed if your server does any disk access, otherwise it's mostly about network speed. It's 2022 so network speed will rarely be what's limiting your app anymore, so unless you're doing disk access, then ignore this point.
• Number of connections supported by your OS. Every OS has a built-in hard limit (the maximum that cannot be changed) and sometimes also a soft limit (a default limit which can be increased).

Estimating capacity

Your dev machine should, theoretically, be slower than your production server, so you can get a lower bound on the capacity of your server by load testing it. You can use tools like Autocannon or Loadtest to ballpark your capacity. If you start with only a few simultaneous connections and ramp up, you should reach a point where you see the latency suddenly increase (latency should be more or less consistent until then). This is when you are starting to hit a limit.

Expanding your capacity

Threadpool

Node.js is single-threaded, but asynchronous calls run in the Lib UV thread pool. When Node.js is waiting on IO, there is a LibUV thread spinning behind the scenes. When the Lib UV thread pool is full, Node.js has to wait for another to become available before another async IO task can be started, which slows everything down. The default thread pool size in Node.js is quite small (used to be 4), so increasing it can be quite beneficial. You can find more information on tuning the LibUV threadpool size here and here.

Other concerns

Because you specifically mentioned Docker in your question, remember that Docker is only a deployment strategy, and does not by itself help alleviate any load. If you're bound by a threadpool limit, then load balancing to multiple Docker instances on the same machine will speed up your process until you hit one of the other caps. If you're already CPU-bound or IO-bound, then multiple instances running on the same server won't help. At this point you'll need to either vertically scale your server machine or add more machines.