I'm unable to run my testng script using maven. It says build successful each time when I attempt to run mvn test -Dtest com.example.tests.ScriptTest();
pom.xml
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>3.2.1</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.6.1</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
Test file
package com.example.tests;
import org.testng.annotations.Test;
public class ScriptTest{
@Test
public void testScript() {
System.out.println(“This is a test script”);
}
}
In .gitlab-ci I'm trying to run the maven test.
image: maven:latest
stages:
- test
test:
stage: test
script:
- mvn test -Dtest=com.example.tests.ScriptTest
artifacts:
paths:
- target/sunfire-reports
Here is an update showing the entire pom file.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.2.0-SNAPSHOT</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>scanner</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>war</packaging>
<name>scanner</name>
<description>Scan System</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc11</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>net.sourceforge.tess4j</groupId>
<artifactId>tess4j</artifactId>
<version>4.5.4</version>
</dependency>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>3.2.1</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.6.1</version>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.pdfbox</groupId>
<artifactId>pdfbox</artifactId>
<version>2.0.31</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<releases>
<enabled>false</enabled>
</releases>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
<pluginRepository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<releases>
<enabled>false</enabled>
</releases>
</pluginRepository>
</pluginRepositories>
</project>
Expected desired output: The test should execute within gitlab ci
The root cause of the problem is that the surefire plugin that gets resolved from the super pom file is finding JUnit5 bindings first.
Latest Spring provided parent pom file, always brings in JUnit5 dependencies.
The way surefire plugin works is by scanning the classpath to see which test runner is present. AFAIK, the moment it finds a test runner (JUnit or TestNG) it stops scanning the classpath. Since surefire plugin always finds the JUnit5 dependencies (Via the spring parent pom file) it always tries to run JUnit5 test cases.
If you run the command mvn dependency:tree | grep junit you can see an output like below which confirms what I am saying in terms of JUnit5 dependencies being available on the classpath.
➜ mvn dependency:tree | grep junit
[INFO] | +- org.junit.jupiter:junit-jupiter:jar:5.10.1:test
[INFO] | | +- org.junit.jupiter:junit-jupiter-api:jar:5.10.1:test
[INFO] | | | +- org.junit.platform:junit-platform-commons:jar:1.10.1:test
[INFO] | | +- org.junit.jupiter:junit-jupiter-params:jar:5.10.1:test
[INFO] | | \- org.junit.jupiter:junit-jupiter-engine:jar:5.10.1:test
[INFO] | | \- org.junit.platform:junit-platform-engine:jar:1.10.1:test
[INFO] | +- org.mockito:mockito-junit-jupiter:jar:5.7.0:test
When you run the command mvn test -Dtest=com.example.tests.ScriptTest to run your test you will notice something like below in your terminal
[INFO] --- surefire:3.1.2:test (default-test) @ qn_78271590 ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.514 s
[INFO] Finished at: 2024-04-07T09:19:34+05:30
[INFO] ------------------------------------------------------------------------
This above output confirms that surefire plugin is defaulting to the provider (imagine it to be a means to run a JUnit test) org.apache.maven.surefire.junitplatform.JUnitPlatformProvider. This provider knows ONLY how to look for JUnit tests and since it doesn't find any, you end up with nothing being executed.
Now to fix this, you will need to force surefire plugin to always use the TestNG provider.
You can do that, by adding an explicit reference to surefire plugin, that looks like below in your <plugins> section of your pom file.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<!-- ensure that this version always matches
with the surefire plugin version
to ensure you never face any discrepancies.
Use 'mvn help:effective-pom' to list plugins with
their versions being used in your project.
-->
<version>3.1.2</version>
</dependency>
</dependencies>
</plugin>
Now when you run your test again using the command mvn test -Dtest=com.example.tests.ScriptTest you will see an output that looks like below:
[INFO] --- surefire:3.1.2:test (default-test) @ scanner ---
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.example.tests.ScriptTest
This is a test script
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.340 s -- in com.example.tests.ScriptTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.838 s
[INFO] Finished at: 2024-04-07T09:24:55+05:30
[INFO] ------------------------------------------------------------------------
Notice how surefire plugin now uses the TestNG provider to run the tests from the line Using configured provider org.apache.maven.surefire.testng.TestNGProvider in the above output.