Search code examples
mavenmaven-surefire-plugin

How Does surefire decide on the Test framework?


I have been trying to understand how Surefire plugin internally decides which Testing Framework to use ( TestNG, Jupiter, Junit4 etc. )

Does it use reflection and try to find the presence of each framework in the classpath ?
( Looking at the dependencies, Surefire seems to be coming with junit4 in its transitive dependencies - junit:JUnit:jar:4.12 )


Solution

  • It's possible to pass the provider (test-framework type) explicitly, setting an additional plugin dependency, e.g. for TestNG:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.0.0-M5</version>
        <dependencies>
          <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>3.0.0-M5</version>
          </dependency>
        </dependencies>
      </plugin>
    

    If nothing specified

    Surefire normally automatically selects which test-framework provider to use based on the version of TestNG/JUnit present in your project's classpath.

    From this doc:

    https://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html

    How Surefire plugin internally decides which Testing Framework to use

    Let's look at how it's implemented.

    There is ProviderInfo interface with method boolean isApplicable();

    ProviderInfo.java

    I've found multiple implementations in class AbstractSurefireMojo.java

    AbstractSurefireMojo.java

    for:

    • TestNgProviderInfo
    • JUnit3ProviderInfo
    • JUnit4ProviderInfo
    • JUnitPlatformProviderInfo
    • JUnitCoreProviderInfo
    • DynamicProviderInfo

    And there is also a protected method protected List<ProviderInfo> createProviders( TestClassPath testClasspath ) which reference all this implementations.

    protected List<ProviderInfo> createProviders( TestClassPath testClasspath )
            throws MojoExecutionException
        {
            Artifact junitDepArtifact = getJunitDepArtifact();
            return providerDetector.resolve( new DynamicProviderInfo( null ),
                new JUnitPlatformProviderInfo( getJUnit5Artifact(), testClasspath ),
                new TestNgProviderInfo( getTestNgArtifact() ),
                new JUnitCoreProviderInfo( getJunitArtifact(), junitDepArtifact ),
                new JUnit4ProviderInfo( getJunitArtifact(), junitDepArtifact ),
                new JUnit3ProviderInfo() );
        }
    

    and ProviderDetector class invokes isApplicable() per each providerInfo in resolve method.

    ProviderDetector.java

    And looks like the first applicable is selected:

    private Optional<ProviderInfo> autoDetectOneWellKnownProvider( ProviderInfo... wellKnownProviders )
        {
            Optional<ProviderInfo> providerInfo = stream( wellKnownProviders )
                .filter( ProviderInfo::isApplicable )
                .findFirst();
    
            providerInfo.ifPresent( p -> logger.info( "Using auto detected provider " + p.getProviderName() ) );
    
            return providerInfo;
        }