Unable to force test timeout for Cucumber/Playwright tests ran with JUnit5

Question

I am using Playwright in combination with Cucumber. All of my tests are ran in parallel through maven surefire with the JUnit 5 test runner. The issue I am having is that I have been unable to enforce a timeout for my tests, so sometimes a single test can get stuck in CI and run for hours before we notice and have to cancel the build.

I have the following junit-platform.properties file which controls how the cucumber tests are run:

cucumber.execution.parallel.enabled=true
cucumber.execution.parallel.config.strategy=dynamic
cucumber.execution.parallel.config.dynamic.factor=1
cucumber.plugin=formatters.SummaryFormatter,pretty,junit:target/junitreport.xml,json:target/cucumber-report/cucumber.json
cucumber.execution.exclusive-resources.isolated.read-write=org.junit.platform.engine.support.hierarchical.ExclusiveResource.GLOBAL_KEY
cucumber.junit-platform.naming-strategy=long

I have the following Cucumber Runner:

@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("features")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "stepdefinitions")
@SuppressWarnings({"java:S2187", "checkstyle:AbbreviationAsWordInName"})
public class CucumberRunnerIT {

  private CucumberRunnerIT() {
  }
}

I have tried a variety of different solution that I've read online, however none of them worked, the tests continue to run even once the timeout value has been reached.

1: Using the @Timeout annotation and adding this to my CucumberRunnerIT

@Timeout(value = 10, unit = TimeUnit.MINUTES)

2: Configuring a timeout in the junit-platform.properties:

junit.jupiter.execution.timeout.default=10m

3: Configuring the timeout in the POM:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M5</version>
    <configuration>
        <systemPropertyVariables>
            <junit.jupiter.execution.timeout.default>10m</junit.jupiter.execution.timeout.default>
        </systemPropertyVariables>
    </configuration>
</plugin>

Regardless of any solution, the timeout is not respected.

Answer 1

The JUnit User Guide explains:

Unlike previous versions of JUnit, JUnit 5 is composed of several different modules from three different sub-projects.

JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

The JUnit Platform serves as a foundation for launching testing frameworks on the JVM. It also defines the TestEngine API for developing a testing framework that runs on the platform.

And like JUnit Jupiter and JUnit Vintage, Cucumber is an Engine implemented using the JUnit Platform.

The property junit.jupiter.execution.timeout.default=10m configures the timeout for JUnit Jupiter. This has no effect on Cucumber. The @org.junit.jupiter.api.Timeout annotation also does not work for the same reason.

Cucumber currently does not support time outs. They're actually quite complicated.