Search code examples
javamavenjunit5maven-surefire-plugin

Correctly include jUnit5 @Nested classes when running single class in maven

The @Nested classes that are executed in JUnit 5 they are ordered to run AFTER all the tests in eclosing class. How could I enforce the same behavior using maven, if my goal is to run a single enclosing class and it's nested classes? Is there a commandline or pom.xml modification to make this example test pass?

package example;

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;

public class SomeTopLevelTest {

    private static boolean nestedDone = false;

    @Test
    void test1() {
        Assertions.assertFalse(nestedDone, 
            "Nested classes should execute after the enclosing tests");
    }

    @Nested
    class SomeNestedTest {
        @Test
        void test2() {
            nestedDone = true;
        }
    }

    @AfterAll
    static void recheck() {
        Assertions.assertTrue(nestedDone, "Nested class should be executed");
    }
}

This does pass in IDE:
enter image description here

But does not in commanline, if I try to specify the name:

mvn test -Dtest=example.SomeTopLevelTest

[ERROR] Failures: 
[ERROR]   SomeTopLevelTest.recheck:27 Nested class should be executed ==> expected: <true> but was: <false>


mvn test -Dtest=example.SomeTopLevelTest*
[ERROR] Failures: 
[ERROR]   SomeTopLevelTest.test1:14 Nested classes should execute after the enclosing tests ==> expected: <false> but was: <true>
Solution

  • The problem of @Nested classes not executing is a known issue and it has been reported in both JUnit5 and Surefire issue trackers, but as of now remains unresolved.

    The current state of affairs (tested with Maven 3.6.3, Junit5 5.7.2, Surefire 2.22.2 up to 3.0.0-M5):

    A. Not selecting a test

    mvn test

    Results in executing all test classes as expected: methods from enclosing classes first, and then methods from @Nested classes, level by level

    B. Selecting test the standard way

    mvn test -Dtest=example.SomeTopLevelTest

    This apparently triggers the default surefire excludes that use the following pattern:

    <excludes>
    <exclude>**/*$*</exclude>
</excludes>

    Why does it not happen in case A is a mystery, but one can override this behaviour by explicitly clearing the excludes pattern:

        <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
            <excludes>
                <exclude/>
            </excludes>
        </configuration>
    </plugin>

    It does not seem to be possible to do without modyfing the pom.xml.
    This DOES NOT solve the issue as posted in this question, because the nested classes are still executed first.

    C. Using wildcard with -Dtest parameter

    mvn test -Dtest=example.SomeTopLevelTest*

    This explicitly selects all the nested classes, but - as stated in the question - results in executing the nested classes first, so it's not a solution.

    D. Using includes

    mvn test -DtestName=example.SomeTopLevelTest

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <includes>
            <include>${testName}</include>
        </includes>
    </configuration>
</plugin>

    Apparently include patterns work quite differently than -Dtest parameter, because this is finally the solution to pass the test from the question. With this setup the testName may be a single class, wildcard pattern or regex

    • example.SomeTopLevelTest to execute all test methods in single class
    • example/* - all tests (including nested) in package example, but not sub-packages
    • example/** - all tests in package and subpackages
    • advanced regex, is supprted too