Problems running both JUnit tests and Selenium/Cucumber tests at the same time

Question

I have a basic Maven/Spring test project with some simple JUnit tests and Selenium/Cucumber tests that I later want to run using Github Actions, but first I want to make sure they run locally.

This is the JUnit "test":

package com.example.CucumberSelenium;

import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;

import static org.junit.jupiter.api.Assertions.assertTrue;

@SpringBootTest
public class GithubActionsTests {

    @Test
    public void test() {
        assertTrue(true);
    }
}

This is my Cucumber test runner file:

package com.example.CucumberSelenium;

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;
import org.springframework.boot.test.context.SpringBootTest;

@RunWith(Cucumber.class)
@CucumberOptions(features = "src/test/java/com/example/CucumberSelenium/resources/features", glue = "com.example.CucumberSelenium.stepdefs")
public class CucumberTestRunnerTest {
}

This is my pom.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.4</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>CucumberSelenium</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>CucumberSelenium</name>
    <description>testing</description>
    <properties>
        <java.version>21</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <scope>test</scope>
            <version>7.11.2</version>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>7.11.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.17.0</version>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <includes>
                        <include>**/*Tests.java</include>
                        <include>**/*Test.java</include>
                        <include>**/GithubActionsTests.java</include>
                        <include>**/CucumberTestRunnerTest.java</include>
                    </includes>
                </configuration>
            </plugin>

        </plugins>
    </build>
</project>

This is my file structure:

Both tests run fine by them themselves using IntelliJ. But when running mvn test only the cucumber tests are running.

Changing maven-surefire-plugin version to 2.22.2 will reverse the result, only JUnit test is running and not the cucumber tests. So I guess there is some dependecy issue or issue with the plugin, but I can't figure out what. Please advise 🙏

Answer 1

You are using Spring Boot 3.2 which pulls in JUnit 5. While also using cucumber-junit which depends on JUnit 4.

Support for JUnit 5 was introduced into Surefire somewhere around v2.22. But Surefire can not run JUnit 4 and JUnit 5 together at the same time and so defaults to the latest version of JUnit when both are available.

For the short term you can work around this by including the junit-vintage-engine

<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <scope>test</scope>
</dependency>

For the long term it would be better to stop depending on JUnit 4 all together and switch to the cucumber-junit-platform-engine. You can find a working example (without Spring) in the https://github.com/cucumber/cucumber-java-skeleton.