Running tests with cucumber-junit-platform-engine and Selenium WebDriver opens too many threads

Question

I have tried to configure an existing Maven project to run using cucumber-junit-platform-engine.

I have used this repo as inspiration.

I added the Maven dependencies needed, as in the linked project using spring-boot-starter-parent version 2.4.5 and cucumber-jvm version 6.10.4.

I set the junit-platform properties as follows:

cucumber.execution.parallel.enabled=true
cucumber.execution.parallel.config.strategy=fixed
cucumber.execution.parallel.config.fixed.parallelism=4

Used annotation @Cucumber in the runner class and @SpringBootTest for classes with steps definition.

It seems to work fine with creating parallel threads, but the problem is it creates all the threads at the start and opens as many browser windows (drivers) as the number of scenarios (e.g. 51 instead of 4).

I am using a CucumberHooks class to add logic before and after scenarios and I'm guessing it interferes with the runner because of the annotations I'm using:

import java.util.List;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;

import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;
import io.cucumber.plugin.ConcurrentEventListener;
import io.cucumber.plugin.event.EventHandler;
import io.cucumber.plugin.event.EventPublisher;
import io.cucumber.plugin.event.TestRunFinished;
import io.cucumber.plugin.event.TestRunStarted;
import io.github.bonigarcia.wdm.WebDriverManager;

public class CucumberHooks implements ConcurrentEventListener {

@Autowired
private ScenarioContext scenarioContext;

@Before
public void beforeScenario(Scenario scenario) {
    scenarioContext.getNewDriverInstance();
    scenarioContext.setScenario(scenario);
    LOGGER.info("Driver initialized for scenario - {}", scenario.getName());
    ....
    <some business logic here>
    ....
}

@After
public void afterScenario() {
    Scenario scenario = scenarioContext.getScenario();
    WebDriver driver = scenarioContext.getDriver();

    takeErrorScreenshot(scenario, driver);
    LOGGER.info("Driver will close for scenario - {}", scenario.getName());

    driver.quit();
}

private void takeErrorScreenshot(Scenario scenario, WebDriver driver) {
    if (scenario.isFailed()) {
        final byte[] screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);
        scenario.attach(screenshot, "image/png", "Failure");
    }
}

@Override
public void setEventPublisher(EventPublisher eventPublisher) {
    eventPublisher.registerHandlerFor(TestRunStarted.class, beforeAll);
}

private EventHandler<TestRunStarted> beforeAll = event -> {
    // something that needs doing before everything
    .....<some business logic here>....
    WebDriverManager.getInstance(DriverManagerType.CHROME).setup();
};

}

I tried replacing the @Before tag from io.cucumber.java with the @BeforeEach from org.junit.jupiter.api and it does not work.

How can I solve this issue?

Answer 1

So as it turns out parallism is mostly a suggestion. Cucumber uses JUnit5s ForkJoinPoolHierarchicalTestExecutorService which constructs a ForkJoinPool.

From the docs on ForkJoinPool:

For applications that require separate or custom pools, a ForkJoinPool may be constructed with a given target parallelism level; by default, equal to the number of available processors. The pool attempts to maintain enough active (or available) threads by dynamically adding, suspending, or resuming internal worker threads, even if some tasks are stalled waiting to join others. However, no such adjustments are guaranteed in the face of blocked I/O or other unmanaged synchronization.

So within a ForkJoinPool when ever a thread blocks for example because it starts asynchronous communication with the web driver another thread may be started to maintain the parallelism.

Since all threads wait, more threads are added to the pool and more web drivers are started.

This means that rather then relying on the ForkJoinPool to limit the number of webdrivers you have to do this yourself. You can use a library like Apache Commons Pool or implement a rudimentary pool using a counting semaphore.

@Component
@ScenarioScope
public class ScenarioContext {

    private static final int MAX_CONCURRENT_WEB_DRIVERS = 1;
    private static final Semaphore semaphore = new Semaphore(MAX_CONCURRENT_WEB_DRIVERS, true);

    private WebDriver driver;

    public WebDriver getDriver() {
        if (driver != null) {
            return driver;
        }

        try {
            semaphore.acquire();
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }

        try {
            driver = CustomChromeDriver.getInstance();
        } catch (Throwable t){
            semaphore.release();
            throw t;
        }
        return driver;
    }

    public void retireDriver() {
        if (driver == null) {
            return;
        }

        try {
            driver.quit();
        } finally {
            driver = null;
            semaphore.release();
        }
    }
}