Maven Surefire Double-Counting Tests
While not terribly well-versed in Maven lifecycles, I have read a fair amount of documentation over the last ~three days and found nothing showing a relationship between build cycles, goals, or my plugins and the issue I'm seeing.
I'm experiencing a weird issue with the latest Surefire plugin (3.0.0-M1) where my unit tests are completing successfully but appear to be running twice (or maybe are just being counted twice?).
Some context - before I specified any plugin or version in this POM file, Surefire ran on its own with a default version using:
mvn clean package
I have no idea why. And I never really stopped to ask why that might be. It just always has so I took it for granted and now it's behaving weird. I probably deserve this.
This is the POM file for that run (with some properties and distribution removed):
<build>
<sourceDirectory>src/main/java</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<configuration>
<destFile>${basedir}/target/jacoco.exec</destFile>
<dataFile>${basedir}/target/jacoco.exec</dataFile>
</configuration>
<executions>
<execution>
<id>jacoco-initialize</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>jacoco-site</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>true</createDependencyReducedPom>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.capitalone.cardcore.App</mainClass>
</transformer>
</transformers>
<artifactSet>
<excludes>
<artifact>junit:*</artifact>
<artifact>org.hamcrest:*</artifact>
<artifact>org.junit.jupiter:*</artifact>
<artifact>org.apiguardian:*</artifact>
<artifact>org.powermock:*</artifact>
<artifact>org.mockito:*</artifact>
<artifact>commons-beanutils:*</artifact>
<artifact>javax.xml.*</artifact>
<artifact>org.apache.hadoop:hadoop-yarn-client:*</artifact>
<artifact>org.apache.hadoop:hadoop-yarn-server-common:*</artifact>
<artifact>org.apache.hadoop:hadoop-yarn-api:*</artifact>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>**/*.dll</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.6</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.9</version>
</dependency>
<!-- HTTP -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.4</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<dependency>
<groupId>com.google.http-client</groupId>
<artifactId>google-http-client</artifactId>
<version>1.23.0</version>
</dependency>
<dependency>
<groupId>com.google.http-client</groupId>
<artifactId>google-http-client-jackson2</artifactId>
<version>1.23.0</version>
</dependency>
<!-- JRECORD -->
<dependency>
<groupId>net.sf.JRecord</groupId>
<artifactId>JRecord</artifactId>
<version>0.81.1</version>
</dependency>
<dependency>
<groupId>net.sf.bruce_a_martin.cb2xml</groupId>
<artifactId>cb2xml</artifactId>
<version>0.95.3</version>
</dependency>
<!-- PGP -->
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpg-jdk15on</artifactId>
<version>1.47</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk15on</artifactId>
<version>1.47</version>
</dependency>
<!-- UNIT TESTS -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.0.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-core</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-easymock</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>2.0.2-beta</version>
<scope>test</scope>
</dependency>
<!-- PARQUET OUTPUT -->
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-avro</artifactId>
<version>${parquet.version}</version>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-common</artifactId>
<version>${parquet.version}</version>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-encoding</artifactId>
<version>${parquet.version}</version>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-column</artifactId>
<version>${parquet.version}</version>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-hadoop</artifactId>
<version>${parquet.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>${hadoop.version}</version>
</dependency>
</dependencies>
So - no mention of the surefire plugin, but it ran my tests anyway. Runs all the tests and gives me back 144 as the count (which is correct). It's been like this for over a year, but I added a few code tweaks and now I need to include it as a plugin because I need a few environment variables for my tests like so:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M1</version>
<configuration>
<environmentVariables>
<env>dev</env>
<ENV>DEV</ENV>
</environmentVariables>
</configuration>
</plugin>
But now I get this when I run my test suite:
The first question is: am I losing my mind? When I don't include the plugin, the surefire plugin is running by default when I do nothing (I don't even know how to turn it off), and when I include it, it appears to run twice. Is one of my other plugins running it without broadcasting it? That's a question I'm not proud of. But then - if I keep the plugin included in my POM as a line-item but set it to that "default" version:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<environmentVariables>
<env>dev</env>
<ENV>DEV</ENV>
</environmentVariables>
</configuration>
</plugin>
I get my expected 144 count back and my tests run because they have the variables they need. So is there maybe a problem with the latest version of surefire? Or just the way I'm including it with other packages?
Anyone encountered this with this new Surefire? Or maybe this combination of plugins?
EDIT #1 - 11/18/2018
Including my TestSuite class:
package com.business.cardcore.tests;
import com.business.cardcore.tests.classes.app.*;
import com.business.cardcore.tests.classes.librarytools.*;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
@RunWith(Suite.class)
// run the test classes
@Suite.SuiteClasses({
// App
AppCoreTests.class,
AppFileMapperErrorTests.class,
AppFileProcessorErrorTests.class,
AppSchemaWriterErrorTests.class,
// LibraryTools
ChamberOfSecretsTests.class,
ConfigurationTests.class,
CopybookReaderErrorTests.class,
CopybookReaderTests.class,
DatawiseTests.class,
FileMapperTests.class,
HttpWrapperTests.class,
OutputAsciiTests.class,
OutputEbcdicTests.class,
OutputParquetTests.class,
PGPDecryptTests.class,
SchemaWriterTests.class,
TokenValidationTests.class,
TuringTests.class,
ValidationTests.class
})
public class TestSuite { }
See the References section of this answer to Maven: Lifecycle vs. Phase vs. Plugin vs. Goal for why plugins run during a Maven build without having declared them.
A wild guess concerning your doubled tests: In your first image we can see that you're running a TestSuite. Maybe your tests are run (or counted) once via this suite and once individually. What if you remove/disable the suite.
- org.hibernate.query.SemanticException: Could not resolve attribute
- Can I delay a stubbed method response with Mockito?
- How can I combine instanceof and cast in Java?
- Hibernate: flush() and commit()
- Java GMT display
- Retrieve result of StoredProcedure that has NO out parameter
- Jena's getLocalName doesn't return numeric localname with Turtle
- Parse meta tag and get HTML content from body with Tika
- Best way to compare dates in Android
- Merging cells in Excel using Apache POI
- Months elapsed function not giving proper result
- Unit test IOException with Java, JUnit and Mockito
- How to handle asynchronous thread processing with same transaction in SpringBoot?
- Navigating the file system with netbeans is VERY slow
- Spring caching works "live" but not in tests
- How can I compare two Calendar dates?
- Error: module not found for Apache Commons Math (commons-math3) when using Gradle and Java modules
- Why do Java 2D arrays allow holding rows larger than their specified size?
- Flattening nested attributes in Jackson
- org.apache.commons.logging.LogFactory was unintentionally initialized at build time
- NavigationView menu items with actionLayout. How can I set those action layout's attribute for each item?
- Is there a difference between single and double quotes in Java?
- Process, map, and collect objects in one stream call
- Build jvm without jdk
- java stream Collectors without mapper?
- Adding Subtype to generic List
- How to read a text-file resource into Java unit test?
- Strange behaviour of ww SimpleDateFormat
- How to convert a java Map to immutable Scala map via java code?
- Multiple buttons in single page navigating each button to different page in android studio