Execution of Tasks in ExecutorService without Thread pauses

Question

I have a thread pool with 8 threads

private static final ExecutorService SERVICE = Executors.newFixedThreadPool(8);

My mechanism emulating the work of 100 user (100 Tasks):

List<Callable<Boolean>> callableTasks = new ArrayList<>();
for (int i = 0; i < 100; i++) { // Number of users == 100
    callableTasks.add(new Task(client));
}
SERVICE.invokeAll(callableTasks);
SERVICE.shutdown();

The user performs the Task of generating a document.

1. Get UUID of Task;
2. Get Task status every 10 seconds;
3. If Task is ready get document.

public class Task implements Callable<Boolean> {

    private final ReportClient client;

    public Task(ReportClient client) {
        this.client = client;
    }

    @Override
    public Boolean call() {
        final var uuid = client.createDocument(documentId);
        GetStatusResponse status = null;
        do {
            try {
                Thread.sleep(10000); // This stop current thread, but not a Task!!!!
            } catch (InterruptedException e) {
                return Boolean.FALSE;
            }
            status = client.getStatus(uuid);
        } while (Status.PENDING.equals(status.status()));
        final var document = client.getReport(uuid);
        return Boolean.TRUE;
    }
}

I want to give the idle time (10 seconds) to another task. But when the command Thread.sleep(10000); is called, the current thread suspends its execution. First 8 Tasks are suspended and 92 Tasks are pending 10 seconds. How can I do 100 Tasks in progress at the same time?

Answer 1

EDIT: I just posted this answer and realized that you seem to be using that code to emulate real user interactions with some system. I would strongly recommend just using a load testing utility for that, rather than trying to come up with your own. However, in that case just using a CachedThreadPool might do the trick, although probably not a very robust or scalable solution.

Thread.sleep() behavior here is working as intended: it suspends the thread to let the CPU execute other threads. Note that in this state a thread can be interrupted for a number of reasons unrelated to your code, and in that case your Task returns false: I'm assuming you actually have some retry logic down the line.

So you want two mutually exclusive things: on the one hand, if the document isn't ready, the thread should be free to do something else, but should somehow return and check that document's status again in 10 seconds.

That means you have to choose:

• You definitely need that once-every-10-seconds check for each document - in that case, maybe use a cachedThreadPool and have it generate as many threads as necessary, just keep in mind that you'll carry the overhead for numerous threads doing virtually nothing.

• Or, you can first initiate that asynchronous document creation process and then only check for status in your callables, retrying as needed.

Something like:

public class Task implements Callable<Boolean> {
    private final ReportClient client;
    private final UUID uuid;
    // all args constructor omitted for brevity
    @Override
    public Boolean call() {
        GetStatusResponse status = client.getStatus(uuid);
        if (Status.PENDING.equals(status.status())) {
            final var document = client.getReport(uuid);
            return Boolean.TRUE;
        } else {
            return Boolean.FALSE; //retry next time
        }
    }
}

List<Callable<Boolean>> callableTasks = new ArrayList<>();
for (int i = 0; i < 100; i++) { 
    var uuid = client.createDocument(documentId); //not sure where documentId comes from here in your code
    callableTasks.add(new Task(client, uuid));
}

List<Future<Boolean>> results = SERVICE.invokeAll(callableTasks); 
// retry logic until all results come back as `true` here

This assumes that createDocument is relatively efficient, but that stage can be parallelized just as well, you just need to use a separate list of Runnable tasks and invoke them using the executor service. Note that we also assume that the document's status will indeed eventually change to something other than PENDING, and that might very well not be the case. You might want to have a timeout for retries.