Search code examples
javajava-17

How to avoid RejectedExecutionException with java.net HttpClient after upgrade to Java 17?


I am upgrading a project from Java 11 to Java 17. This project uses java.net.http.HttpClient in several places which after the upgrade run into the following exception:

    ...
Caused by: java.util.concurrent.RejectedExecutionException: Thread limit exceeded replacing blocked worker
    at java.base/java.util.concurrent.ForkJoinPool.tryCompensate(ForkJoinPool.java:1819)
    at java.base/java.util.concurrent.ForkJoinPool.compensatedBlock(ForkJoinPool.java:3446)
    at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3432)
    at java.base/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1898)
    at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2072)
    at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:553)
    at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:123)
    ...

From what I understand the problem is that HttpClient uses CompletableFuture which calls ForkJoinPool.managedBlock(q); in line 1898. This uses the static common pool of ForkJoinPool (correct me if I'm wrong) and I don't have any chance to tell my application to create a separate pool for these calls (or do I?). Furthermore in Java 17 the behavior changed how java deals with the situation when all threads of the pool are in use - which now leads to this exception.

Is there a way around this problem? I already played around with the system parameters java.util.concurrent.ForkJoinPool.common.parallelism and ...common.maximumSpares but without the desired result.

What can I do to avoid this exception? Thanks for all your help!


Solution

  • It turned out that in our case we used a ForkJoinPool around all this and we had to configure its saturate predicate:

    This all happened in our JUnit5 tests where we use a custom ParallelExecutionConfiguration. This executes our tests in parallel by the use of ForkJoinPool.

    Luckily the mentioned interface provides a new method getSaturatePredicate() starting with JUnit 5.9.0. So all we had to do is to upgrade JUnit5 and implement this method:

    @Override
    public Predicate<? super ForkJoinPool> getSaturatePredicate() {
        return forkJoinPool -> true;
    }