Multiple CompletionService for one thread pool Java

Question

I'm working on a Java server application with the general following architecture:

• Clients make RPC requests to the server
• The RPC server (gRPC) I believe has its own thread pool for handling requests
• Requests are immediately inserted into Thread Pool 1 for more processing
• A specific request type, we'll call Request R, needs to run a few asynchronous tasks in parallel, judging the results to form a consensus that it will return to the client. These tasks are a bit more long running, so I use a separate Thread Pool 2 to handle these requests. Importantly, each Request R will need to run the same 2-3 asynchronous tasks. Thread Pool 2 therefore services ALL currently executing Request R's. However, a Request R should only be able to see and retrieve the asynchronus tasks that belong to it.
• To achieve this, upon every incoming Request R, while its in Thread Pool 1, it will create a new CompletionService for the request, backed by Thread Pool 2. It will submit 2-3 async tasks, and retrieve the results. These should be strictly isolated from anything else that might be running in Thread Pool 2 belonging to other requests.
• My questions:
  • Firstly, is Java's CompletionService isolated? I couldn't find good documentation on this after checking the JavaDocs. In other words, if two or more CompletionService's are backed by the same thread pool, are any of them at risk of pulling a future belonging to another CompletionService?
  • Secondly, is this bad practice to be creating this many CompletionService's for each request? Is there a better way to handle this? Of course it would be a bad idea to create a new thread pool for each request, so is there a more canonical/correct way to isolate futures within a CompletionService or is what I'm doing okay?

Thanks in advance for the help. Any pointers to helpful documentation or examles would be greatly appreciated.

Code, for reference, although trivial:

public static final ExecutorService THREAD_POOL_2 =
        new ThreadPoolExecutor(16, 64, 60, TimeUnit.SECONDS, new LinkedBlockingQueue<>());

// Gets created to handle a RequestR, RequestRHandler is run in Thread Pool 1
public class RequestRHandler {

    CompletionService<String> cs;

    RequestRHandler() {
        cs = new ExecutorCompletionService<>(THREAD_POOL_2);
    }

    String execute() {
        cs.submit(asyncTask1);
        cs.submit(asyncTask2);
        cs.submit(asyncTask3);

       // Lets say asyncTask3 completes first
      Future<String> asyncTask3Result = cs.take();

      // asyncTask3 result indicates asyncTask1 & asyncTask2 results don't matter, cancel them
      // without checking result


      // Cancels all futures, I track all futures submitted within this request and cancel them,
      // so it shouldn't affect any other requests in the TP 2 pool
      cancelAllFutures(cs);


      return asyncTask3Result.get();  
    }


}

Answer 1

Firstly, is Java's CompletionService isolated?

That's not garanteed as it's an interface, so the implementation decides that. But as the only implementation is ExecutorCompletionService I'd just say the answer is: yes. Every instance of ExecutorCompletionService has internally a BlockingQueue where the finished tasks are queued. Actually, when you call take on the service, it just passes the call to the queue by calling take on it. Every submitted task is wrapped by another object, which puts the task in the queue when it's finished. So each instance manages it's submitted tasks isolated from other instances.

Secondly, is this bad practice to be creating this many CompletionServices for each request?

I'd say it's okay. A CompletionService is nothing but a rather thin wrapper around an executor. You have to live with the "overhead" (internal BlockingQueue and wrapper instances for the tasks) but it's small and you are probably gaining way more from it than it costs. One could ask if you need one for just 2 to 3 tasks but it kinda depends on the tasks. At this point it's a question about if a CompletionService is worth it in general, so that's up to you to decide as it's out of scope of your question.