Search code examples
javamavencucumbertestngmaven-surefire-plugin

Maven Surefire plugin not starting tests defined in TestNG suite XML file

I am developing a test automation project that uses: Maven, TestNG, Cucumber and Selenium WebDriver. My OS is Windows 10 and I'm using IntelliJ as my IDE.

I have a pom.xml file that looks like this (project name is changed for showcase purposes):

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.example</groupId>
<artifactId>project-name</artifactId>
<version>1.0-SNAPSHOT</version>

<dependencies>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>7.3.0</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-java</artifactId>
        <version>6.9.0</version>
    </dependency>

    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-testng</artifactId>
        <version>6.9.0</version>
    </dependency>

    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-picocontainer</artifactId>
        <version>6.9.0</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>io.github.bonigarcia</groupId>
        <artifactId>webdrivermanager</artifactId>
        <version>4.2.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.11</version>
    </dependency>

    <dependency>
        <groupId>org.iban4j</groupId>
        <artifactId>iban4j</artifactId>
        <version>3.2.1</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
                <source>11</source>
                <target>11</target>
                <encoding>UTF-8</encoding>
            </configuration>
        </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.12.4</version>
            <configuration>
                <suiteXmlFiles>
                    <suiteXmlFile>src/test/resources/TestSuites/suite.xml</suiteXmlFile>
                </suiteXmlFiles>
                <testFailureIgnore>true</testFailureIgnore>
            </configuration>
        </plugin>
    </plugins>
</build>

As you can see, path to "suite.xml" is defined in POM and is correct: enter image description here

TestNG suite "suite.xml" file looks like this:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="All Tests Suite" verbose="1">
<test name="Test">
    <classes>
        <class name="Runners.RunnerAllFeatures"/>
    </classes>
</test>
</suite>

Suite file has "RunnerAllFeatures" as it's test class. When I manually start that suite file (right click and "Run") it works well - cucumber scenarios defined in the "RunnerAllFeatures" are being executed. Runner class "RunnerAllFeatures" looks like this:

package Runners;
import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
@CucumberOptions(
    features = {"src/test/resources/Features/completed"},
    glue = {"StepDefinitions"},
    monochrome = true,
    plugin = {"pretty", "html:target/cucumber.html"}
    )
public class RunnerAllFeatures extends AbstractTestNGCucumberTests{}

Problem occurs when I am trying to run "suite.xml" via Maven in terminal. I use "mvn clean test" to do this. The build summary states "BUILD SUCCESS" but no tests were ran. Log line "T E S T S" is not even present. There are no warnings that would be (imo) significant to the problem. Here is one of the log snippets:

    C:\Users\username\IdeaProjects\project-name>mvn clean test
[INFO] Scanning for projects...
[INFO]
[INFO] -------------< org.example:project-name >--------------
[INFO] Building project-name 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ project-name ---
[INFO] Deleting C:\Users\username\IdeaProjects\project-name\target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ project-name ---
[WARNING] Using platform encoding (Cp1250 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 4 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ project-name ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 39 source files to C:\Users\username\IdeaProjects\project-name\target\classes
[INFO] /C:/Users/username/IdeaProjects/project-name/src/main/java/baseMethods/BaseMethods.java: C:\Users\username\IdeaProjects\project-name\src\main\java\baseMethods\BaseMethods.java uses un
checked or unsafe operations.
[INFO] /C:/Users/username/IdeaProjects/project-name/src/main/java/baseMethods/BaseMethods.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ project-name ---
[WARNING] Using platform encoding (Cp1250 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 17 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ project-name ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 17 source files to C:\Users\username\IdeaProjects\project-name\target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ project-name ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.108 s
[INFO] Finished at: 2021-12-29T13:46:30+01:00
[INFO] ------------------------------------------------------------------------

I tried changing backslashes to forward slashes in POM's path to "suite.xml" but it did not change anything. I tried to parameterize that path but the result was the same. I have another test automation project where the only difference in technology stack is the lack of Cucumber. TestNG suite file points to a "@Test" annotated class. Starting that suite file through Maven Surefire works well there.

If there is a need for some more information I will try to provide it.

Solution

  • Found the solution. Runner class name was wrong - it has to include the word "Test". See: https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html

    I was aware of this Maven naming convention but I did not suspect that it might be an issue here since "RunnerAllFeatures" is a runner class and not a test class.