Maven surefire plugin + JUnit5 dynamic tests - how to override test naming?

Question

Assume we have a JUnit5 dynamic test like this:

public class ProbaTest {
    @TestFactory
    public Iterable<DynamicNode> tests() {
        return Collections.singleton(
            DynamicTest.dynamicTest("aaa", () -> {
                throw new AssertionError("FAIL, as planned");
            })
        );
    }
}

When being ran by Surefire Maven plugin, it fails in the following way:

[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ proba-retrolambda ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.proba.ProbaTest
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.025 s <<< FAILURE! - in com.proba.ProbaTest
[ERROR] tests[1]  Time elapsed: 0.007 s  <<< FAILURE!
java.lang.AssertionError: FAIL, as planned
    at com.proba.ProbaTest.lambda$tests$0(ProbaTest.java:14)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   ProbaTest.lambda$tests$0:14 FAIL, as planned
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Notable thing about the output is the dynamic name, given to the test by JUnit5 - "tests[1]". It doesn't use the "aaa" display name, given by the test factory, and as far as I saw, there are reasons for that.

I wonder, however - is there a way of explicitly overriding the naming behavior? Is there any way of how can I provide explicit names for a dynamic JUnit test cases by my own?

Answer 1

One thing I've noticed is that you're using Maven Surefire Plugin version 2.22.2, which is a bit outdated.

As far as I know there is no way of getting the @DisplayName output only by configuration, but since version 3.0.0-M4 there's a way of developing extensions to Maven Surefire, so you can make it deal with the @DisplayName the way you want to for reports and console outputs.

Knowing that, the solution to override the test naming is to develop your own extension or to use an existing one.

I've already followed the above path and developed an extension, so feel free to use it, either as reference to develop a new one or as a possible solution.

Just tried running dynamic tests samples with it:

public class TestTest {

    @TestFactory
    public Iterable<DynamicNode> tests() {
        return Collections.singleton(
                DynamicTest.dynamicTest("aaa", () -> {
                    throw new AssertionError("FAIL, as planned");
                })
        );
    }

    Collection<DynamicTest> tests = Arrays.asList(
            DynamicTest.dynamicTest("Add test",
                    () -> assertEquals(2, Math.addExact(1, 1))),
            DynamicTest.dynamicTest("Multiply Test",
                    () -> assertEquals(4, Math.multiplyExact(2, 2))));

    @TestFactory
    Collection<DynamicTest> dynamicTestsWithCollection() {
        return tests;
    }

    @TestFactory
    @DisplayName("Calculating")
    Collection<DynamicTest> dynamicTestsWithCollectionWithDisplayName() {
        return tests;
    }

}

And got the following output:

[INFO] --- maven-surefire-plugin:3.0.0-M7:test (default-test) @ maven-surefire-junit5-tree-reporter ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] ├─ TestTest - 0.046s
[INFO] │  ├─ ✔ Calculating Add test - 0.008s
[INFO] │  ├─ ✔ Calculating Multiply Test - 0.001s
[INFO] │  ├─ ✘ tests() aaa - 0s
[INFO] │  ├─ ✔ dynamicTestsWithCollection() Add test - 0.001s
[INFO] │  └─ ✔ dynamicTestsWithCollection() Multiply Test - 0.001s
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   TestTest.lambda$tests$0:15 FAIL, as planned
[INFO]
[ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.909 s
[INFO] Finished at: 2022-07-24T14:56:02+02:00
[INFO] ------------------------------------------------------------------------

You can check that the @DisplayName is printed as desired. To try it by yourself, just add this to your pom file:

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M7</version>
    <dependencies>
        <dependency>
            <groupId>me.fabriciorby</groupId>
            <artifactId>maven-surefire-junit5-tree-reporter</artifactId>
            <version>1.1.0</version>
        </dependency>
    </dependencies>
    <configuration>
        <reportFormat>plain</reportFormat>
        <consoleOutputReporter>
            <disable>true</disable>
        </consoleOutputReporter>
        <statelessTestsetInfoReporter
                implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode">
        </statelessTestsetInfoReporter>
    </configuration>
</plugin>