Intermittently getting ParameterResolutionException for JUnit5 parameterized tests running in parallel

I am writing parameterized tests, where test classes and methods are running in parallel. Below is the configuration.

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.mode.classes.default = concurrent
junit.jupiter.execution.parallel.config.dynamic.factor = 2

The tests snippet is below -

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;

@ExtendWith(SimpleCallbackListener.class)
public class HelloWorldTest {
    private static final Map<String, List<TestData>> testDataMap;

    static {
        testDataMap = TestsConfigManager.readTestConfig("test-data");
    }

    private static List<Object[]> data() {
        final List<Object[]> testDataList = prepareTestData(testDataMap);
        return testDataList;
    }

    @ParameterizedTest(name = "{index}: {0}")
    @MethodSource("data")
    @Tag("regression")
    public void testMethodName(
            final String expected,
            final String actual
    ) {
        // test code goes here
    }
}

SimpleCallbackListener is as below -

public class SimpleCallbackListener implements BeforeAllCallback, ExtensionContext.Store.CloseableResource {
    /**
     * gatekeeper to prevent multiple Threads within the same routine
     */
    private static final Lock LOCK = new ReentrantLock();
    private static boolean started = false;
    private Instant startTime;

    @Override
    public void beforeAll(final ExtensionContext context) {
        LOCK.lock();
        try {
            if (!started) {
                startTime = Instant.now();
                requireApplicationInitializer();
                // The following line registers a callback hook when the root test context is shut down
                context.getRoot().getStore(GLOBAL).put(this.getClass().getName(), this);
                started = true;
            }
        } finally {
            LOCK.unlock();
        }
    }
}

Question

This setup is causing below intermittent exception.

org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter ... in method ...
    at java.base@17.0.7/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:194)
    at java.base@17.0.7/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base@17.0.7/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base@17.0.7/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base@17.0.7/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base@17.0.7/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

Looked at answers suggested at -

Solution

I was able to figure out the root cause.

The issue was with the following method.

public static List<Object[]> prepareTestData() {
    final List<Object[]> testDataList = new ArrayList<>();
    
    testDataMap
        .entrySet()
        .parallelStream()
        .forEach(entry -> {
            final String storeAlias = entry.getKey();
            final List<TestData> testDataList = entry.getValue();
            final List<Object[]> list = testDataList
                    .stream()
                    ...
                    .map(testData -> new Object[]{someObj1, someObj2})
                    .toList();
            testDataList.addAll(list);
        });
}

Here, I was trying to add the elements to an ArrayList concurrently. But, then I realized that ArrayList is not thread-safe, and hence this was causing issues.

Solution?

I used CopyOnWriteArrayList.