What happens to incoming requests when a nodejs server is blocked? There are times when the server will be blocked because it is chewing through something computationally expensive, or perhaps doing some synchronous IO (e.g. writing to a sqlite database). This is best described with an example:
given a server like this:
const { execSync } = require('child_process')
const express = require('express')
const app = express()
const port = 3000
// a synchronous (blocking) sleep function
function sleep(ms) {
execSync(`sleep ${ms / 1000}`)
}
app.get('/block', (req, res) => {
sleep(req.query.ms)
res.send(`Process blocked for ${req.query.ms}ms.`)
})
app.get('/time', (req, res) => res.send(new Date()))
app.listen(port, () => console.log(`Example app listening at http://localhost:${port}`))
I can block the nodejs process like so:
# block the server for two seconds
curl http://localhost:3000/block\?ms\=2000
and while it is blocked attempt to make another request to the server:
curl http://localhost:3000/time
the second request will hang until the blocking call is completed, and then respond with the expected datetime. My question is, what specifically is happening to the request while the nodejs process is blocked?
Finally, I understand that blocking nodejs is bad practice but I am not asking about best practices. I want to understand what nodejs does under stressful circumstances like those described here.
In the OS, the TCP stack has a queue for incoming data or connections that is waiting to be picked up by the appropriate host application if the host application is too busy to pick it up right now. Depending upon OS and configuration, that inbound queue will fill up at some point and clients attempting to connect would get an error. I'm not aware of any separate thread in nodejs that picks these up into its own queue and there probably isn't any reason for nodejs to do so as the TCP stack already implements an inbound connection queue on its own.
If you're blocking the nodejs process long enough for 10,000 incoming requests to arrive, you have much bigger problems and need to solve the blocking problem at its core. Nodejs has threads, child processes and clustering all of which can be employed as relief for a blocking calculation.
For data sent on an existing, already-opened TCP connection, there is back pressure (at the TCP level). For new incoming connections, there really isn't such a thing as back pressure. The new incoming connection is either accepted or its not. This is one cause of what we sometimes observe as ERR_CONNECTION_REFUSED.
Some related discussion here: What can be the reason of connection refused errors.
Does node read in the request using some low level c++ and put it into a queue? Is backpressure involved here?
Node itself does not do this (that I'm aware of). The OS TCP stack has a queue for inbound data and incoming connection requests.
Is the unix kernel involved here? Does it know to put a request on some kind of queue while a server refuses to respond?
The TCP stack (in the OS) does have a queue for both incoming data arriving on an existing connection and for inbound connection requests. This queue is of a finite (and partially configurable) size.
Is it just as simple as curl waiting on a response from a socket indefinitely?
No. If the queue for inbound connection requests on the server is full, the connection request will be rejected. If the queue is not full, then it is just a matter of waiting long enough for it to succeed. Most client-side libraries will use some sort of timeout and give up after a time in case something happened that causes there to never be a response sent back.
What happens if the server is blocked and 10,000 new requests hit the server? Will they all be serviced as soon as the server becomes unblocked? (assuming there is no load balancer or other timeout mechanisms in between the client & server)
The target host will queue the inbound connection requests up to some limit (which varies by OS and configuration) and then will reject ones that come after that.
Some other relevant articles:
How TCP backlog works in Linux
What is "backlog" in TCP connections?
TCP Connection Backlog and a Struggling Server
The more of these types of articles you read, the more you will also discover a tradeoff between quickly accepting lots of connections and defending against various types of DOS attacks. It seems a balance has to be drawn.