Why does invokeAll fail with ExecutorService with limited pool size

Question

I have a class that uses an ExecutorService to make url requests, so that they can be run in parallel, and I limit it to a max pool size of 20

private static ExecutorService getCachedPool(ThreadFactory threadFactory)
{
    return new ThreadPoolExecutor(20, 20,
            60L, TimeUnit.SECONDS,
            new SynchronousQueue<Runnable>(),
            threadFactory);
}

However I use it as follows:

List<Future<ArtistCoverImage>> results = ArtistArtworkOnlineLookup.getExecutorService()
    .invokeAll(lookups);

But if I make call and lookups is larger than available pool size it fails. But I dont understand this as the pool size only specifies max threads that can be used, it uses a SynchronousQueue rather than a BlockingQueue so why arem't the extra lookups just added to the queue.

If I simply change to max pool size Integer.MAX_VALUE

private static ExecutorService getCachedPool(ThreadFactory threadFactory)
    {
        return new ThreadPoolExecutor(20, Integer.MAX_VALUE,
                60L, TimeUnit.SECONDS,
                new SynchronousQueue<Runnable>(),
                threadFactory);
    }

there is no problem, but then Im creating rather more pools than I would like. This is a potential problem because Im trying to improve performance on a low powered black box machine and I am trying to minimize extra work.

Answer 1

There seems to be a misconception on your end regarding the capability/behavior of the queue class you picked. That queue isn't unbound or not blocking, to the contrary.

The javadoc tells us:

New tasks submitted in method execute(java.lang.Runnable) will be rejected when the Executor has been shut down, and also when the Executor uses finite bounds for both maximum threads and work queue capacity, and is saturated.

But more importantly, the javadoc for SynchronousQueue says:

A blocking queue in which each insert operation must wait for a corresponding remove operation by another thread, and vice versa.

When you feed 20 long running tasks into it, each one goes to a thread. When they are all still running when #21 comes in, the pool is fully used, and the queue immediately says: "I am full, too". Everything is saturated, the new job isn't accepted any more.

Solution: pick a different kind of queue.