Scaling the maxPoolSize using ThreadPoolExecutor; Why the pool does not increases dynamically its size?

Question

I'm learning about threads, and concurrency package in java as a beginner, and I have gone through the documentation regarding ThreadPoolExecutor to understand the differences between getPoolSize(), getCorePoolSize(), getMaxPoolSize() and tried to implement the same in code.

BIT OF BACKGROUND

From my understanding, Executors.newFixedThreadPool(3) creates a pool with corePoolSize=3 and as we keep executing tasks via pool, threads will be created until 3 and then when the underlying queue size reaches 100 and new tasks are still submitted then that's where maxPoolSize comes into picture and threads are scaled to reach maxPoolSize from corePoolSize now.

public class Test {

    static ThreadPoolExecutor pool=(ThreadPoolExecutor)Executors.newFixedThreadPool(3);
    
    public static void main(String k[]) throws InterruptedException{
        BlockingQueue<Runnable> queue=pool.getQueue();
        pool.execute(()->{
            for(int b=0;b<10;b++)
                System.out.println("Hello "+b);
        });
        pool.execute(()->{
            for(int b=0;b<10;b++)
                System.out.println("Hello "+b);
        });
        pool.setMaximumPoolSize(10);  //Setting maxPoolSize
        
     for(int j=0;j<20000;j++)       
        pool.execute(()->{
            for(int b=0;b<100;b++){
                System.out.println("Hello "+b);
            System.out.println("Queue size "+queue.size()+" "+"and pool size "+pool.getPoolSize()); 
        }
    });
 }

When the above program is executed, I could see the queue size reaching b/w 12000-20000, if this the case then getPoolSize() must print the value greater than corePoolSize as maxPoolSize is set to 10 but every-time it would only print 3 (which is corePoolSize) why does this happen ? As we expect it to scale up to maxPoolSize.

Answer 1

The Executors.newFixedThreadPool returns an ExecutorService; an interface which does not expose (and probably good reason) methods such setMaximumPoolSize and setCorePoolSize.

If you create a pool of type Executors.newFixedThreadPool, this pool should remain fixed during the lifetime of the application. If you want a pool that can adapt its size, accordingly, you should use Executors.newCachedThreadPool() instead.

From my understanding, Executors.newFixedThreadPool(3) creates a pool with corePoolSize-3 (...)

Looking at the implementation of newFixedThreadPool one can see that:

public static ExecutorService newFixedThreadPool(int nThreads) {
    return new ThreadPoolExecutor(nThreads, nThreads,
                                  0L, TimeUnit.MILLISECONDS,
                                  new LinkedBlockingQueue<Runnable>());
}

and

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue) {
    this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
         Executors.defaultThreadFactory(), defaultHandler);
}

so when you pass 3 to the Executors.newFixedThreadPool constructor, you set both the corePoolSize and the maximumPoolSize to 3.

(...) and as we keep executing tasks via pool, threads will be created until 3 and then when the underlying queue size reaches 100 and new tasks are still submitted then that's where maxPoolSize comes into picture and threads are scaled to reach maxPoolSize from corePoolSize now.

Actually, this is not very accurate, but I will explain it in more detail later. For now reading the Java doc of Executors.newFixedThreadPool it states that:

Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue. At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available.

So there is no scaling of the pool (unless you explicitly do so).

When the above program is executed,i could see the queue size reaching b/w 12000-20000, if this the case then getPoolSize() must print the value greater than corePoolSize as maxPoolSize is set to 10 but everytime it would only print 3(which is corePoolSize) why does this happen ? as we expect it to scale upto maxPoolSize.

No, this is not accurate, if you print the pool.getMaximumPoolSize() it will return 10 as expected, calling pool.setMaximumPoolSize(10); will not change the corePoolSize size. However, if you do pool.setCorePoolSize(10); you will increase the pool to be able to handle 10 threads simultaneously.

The this.maximumPoolSize just defines the upper bound of how many threads the pool should handle simultaneously, it does not change the current size of the pool.

Why the pool does not increases dynamically its size?

Looking a little deeper into the newFixedThreadPool implementation, one can see that the Pool is initialized with a task queue new LinkedBlockingQueue<Runnable>() of size equal to Integer.MAX_VALUE. Looking at the method execute one can read in the comments the following:

/*
         * Proceed in 3 steps:
         *
         * 1. If fewer than corePoolSize threads are running, try to
         * start a new thread with the given command as its first
         * task.  The call to addWorker atomically checks runState and
         * workerCount, and so prevents false alarms that would add
         * threads when it shouldn't, by returning false.
         *
         * 2. If a task can be successfully queued, then we still need
         * to double-check whether we should have added a thread
         * (because existing ones died since last checking) or that
         * the pool shut down since entry into this method. So we
         * recheck state and if necessary roll back the enqueuing if
         * stopped, or start a new thread if there are none.
         *
         * 3. If we cannot queue task, then we try to add a new
         * thread.  If it fails, we know we are shut down or saturated
         * and so reject the task.
         */

If one reads carefully points 2 and 3, one can infer that only when a task cannot be added to the queue will the pool create more threads than those specified by the corePoolSize. And since the Executors.newFixedThreadPool uses a queue with Integer.MAX_VALUE you are not able to see the pool dynamically allocating more resources, unless explicitly setting the corePoolSize with pool.setCorePoolSize.

All of this are implementation details that one should not have to care about. Hence, why the Executors interfaces does not expose method such as setMaximumPoolSize.

From ThreadPoolExecutor documentation one can read:

Core and maximum pool sizes

A ThreadPoolExecutor will automatically adjust the pool size (see getPoolSize()) according to the bounds set by corePoolSize (see getCorePoolSize()) and maximumPoolSize (see getMaximumPoolSize()). When a new task is submitted in method execute(java.lang.Runnable), and fewer than corePoolSize threads are running, a new thread is created to handle the request, even if other worker threads are idle. If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full. By setting corePoolSize and maximumPoolSize the same, you create a fixed-size thread pool. By setting maximumPoolSize to an essentially unbounded value such as Integer.MAX_VALUE, you allow the pool to accommodate an arbitrary number of concurrent tasks. Most typically, core and maximum pool sizes are set only upon construction, but they may also be changed dynamically using setCorePoolSize(int) and setMaximumPoolSize(int).

which basically confirms why the pool did not dynamically updated its size.