Reuse of a worker instance to handle several tasks

I have an application for web testing (based on Selenium but this should not matter much here) which executes quite a number of test cases successively. It takes a couple of hours to complete and the number of test cases grows so I would like to use multiple web browser instances to execute several test cases in parallel. The test cases have no dependency on each other.

Very simplified it looks like this:

TestExecutor executor = new TestExecutor(new FirefoxDriver());
for (TestCase test: tests) {
    executor.execute(test);
    // use results here
}

Now I do not know exactly how to parallelize this. I can easily create several TestExecutors connected to several web browsers as Callables and use Executors, CompletitionServices and other nice helper classes but how do I:

pass a new TestCase to a TestExecutor once it is ready with the previous TestCase? The call() method in Callable takes no arguments, so I probably have to implement some setNextTestCase() in my TestExecutor class to achieve this which I don't find really nice. Are there any better alternatives?
reuse the TestExecutor instance for the execution of the next test case? Since every TestExecutor instance requires a web browser instance it takes a long time to initialize it and would result in many windows flashing on the screen if I would create a new TestExecutor for every test case.

Solution

pass a new TestCase to a TestExecutor once it is ready with the previous TestCase?

This is a common question if I'm understanding you. You have a number of threads in a thread-pool but each thread has some context -- in this case a "web browser". You don't want to have to start up a new browser for every job submitted to the thread-pool.

Here are some ideas about how to accomplish this.

Have a BlockingQueue of TestCase objects. Each of your threads then initialize their browser and then dequeue from the queue of TestCase objects until the queue is empty or some shutdown boolean is set to true. You could the submit your TestExecutor objects into a thread-pool, but they would do their own dequeuing of the TestCase objects though your own queue. You would not submit the TestCase to the thread-pool.

BlockingQueue<TestCase> testCaseQueue = new LinkedBlockingQueue<>();
for (TestCase test: tests) {
    testCaseQueue.add(test);
}
// now make your executors and run them in a thread-pool
TestExecutor testExecutor =
     new TestExecutor(new FirefoxDriver(), testCaseQueue);
ExecutorService threadPool = Executors.newCachedThreadPool();
threadPool.submit(testExecutor);
...
// once you've submitted your last executor, you shutdown the pool
threadPool.shutdown();

...
// inside of the executor, they dequeue tests from `testCaseQueue`
while (!shutdown) {
    TestCase testCase = testCaseQueue.poll(0, TimeUnit.MILLISECONDS);
    if (testCase == null) {
       break;
    }
    ...
}

Another idea would be to submit the TestCase to the thread pool and use a ThreadLocal to get the previously configured browser for the test. This is not as optimal because it makes it hard to terminate the browser appropriately when the tests have completed.