Cannot limit concurrent request with Jetty Server Thread Pool max size, why?

Question

I've created a Jetty server with thread pool to limit the number of concurrent request to the server. But the size I specify the total request the I can make is always 5 less than the size I specified.

Dependency:

<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-server</artifactId>
    <version>9.4.6.v20170531</version>
</dependency>

Server Code:

public void httpServer(int port, Handler handler, String name) {
    QueuedThreadPool threadPool = new QueuedThreadPool(10);
    Server server = new Server(threadPool);
    server.setHandler(handler);
    HttpConfiguration http = new HttpConfiguration();

    ServerConnector serverConnector = new ServerConnector(server, new HttpConnectionFactory(http));
    serverConnector.setPort(port);
    server.setConnectors(new Connector[]{serverConnector});

    try {
        log.info(name + " Listener Started on port: " + port);
        server.start();
        server.join();
    } catch (Exception e) {
        log.error("Unable to start Server... Exiting");
        log.error(e, e);
        System.exit(1);
    }
}

Handler:

public void handle(String target, Request baseRequest, HttpServletRequest request, HttpServletResponse response) {
    try {

        BufferedReader bufferedReader = request.getReader();
        String s;
        StringBuilder sb = new StringBuilder();
        PrintWriter writer = response.getWriter();
        while ((s = bufferedReader.readLine()) != null) {
            sb.append(s);
        }
        System.out.println("Got Request");
        Thread.sleep(2000);
        response.setStatus(HttpServletResponse.SC_OK);
        baseRequest.setHandled(true);
    } catch (Exception e) {
        e.printStackTrace();
    }
}

In the above code the ThreadPool size specified is 10. But concurrently I can make only 5. I've tried changing the size to 20, but in that case the concurrent request I can make is always 5 less i.e., 15. Any idea where those 5 threads are getting used?

Answer 1

Jetty doesn't work like that.

ThreadPools don't work like that.

The Jetty ThreadPool is used for anything that needs a thread, not just requests.

That includes :

• low level networking
• nio selectors
• network acceptors
• http session maintenance
• deployment manager behaviors
• working with credential providers
• internal http client behaviors
• dispatched internal requests
• async handling behaviors
• http/2 main connection handling (to pump the protocol to each http/2 session/stream)
• osgi module resolution
• file system monitoring
• timeouts
• annotation / bytecode scanning
• proxy behaviors
• etc...

The demand on your ThreadPool changes depending on what features you use (servlets, bytecode scanning, proxy, fastcgi, etc), what technology you use (eg: http/2, websocket, unixsockets, etc), and even how big your machine is (the number of cores your machine has, nio itself will want threads based on a subset of the number of cores your machine has).

There is also a "Reserved Threads" slice taken out of any provided ThreadPool to handle things that are critical for operations on the server. That reserved threads demand can change during runtime (like above, it also depends on what technologies are in use)

If you want to limit the number of actively being processed requests, use the QoSFilter configured against the url-pattern's that you want to limit. (or use the DoSFilter if you have slightly different requirements)

• https://www.eclipse.org/jetty/documentation/current/qos-filter.html
• https://www.eclipse.org/jetty/documentation/current/dos-filter.html

Don't do it via the ThreadPool, that's never going to work.