Search code examples
javamavenjenkinsmaven-surefire-plugin

Does Jenkins swallow MojoFailureException?

I have configured maven surefire plugin with parameter:

<configuration>
  <forkedProcessTimeoutInSeconds>60</forkedProcessTimeoutInSeconds>
</configuration>

So when test's working more then 60 seconds, the surefire plugin interrupts it.

Everything works perfectly on my local machine when I use mvn test or mvn install, but when I try to build project on Jenkins it just swallows exception, writes into log [ERROR] There was a timeout or other error in the fork and continues the build. As result I get a Finished: SUCCESS message.

Question: Have anyone got this problem? Does anyone know any solution?

Solution

  • One important difference between default options of a maven local build and a Jenkins maven job is that locally the maven.test.failure.ignore option of the Maven Surefire Plugin is set to false (reasonably) so that test failures will also fail the build.

    From official documentation:

    Set this to "true" to ignore a failure during testing. Its use is NOT RECOMMENDED, but quite convenient on occasion.

    However, a Maven Jenkins job will always run setting the same option to true, as such making the Maven build successful even with test failing and turn the status of the Jenkins job to UNSTABLE (and not SUCCESSFUL or FAILED, which may be a point of debate indeed).

    This behavior is also documented in an official Jenkins issue ticket

    Following the Jenkins Terminology, when (surefire or failsafe) tests fail, the Jenkins build status must be UNSTABLE:
    << A build is unstable if it was built successfully and one or more publishers report it unstable. For example if the JUnit publisher is configured and a test fails then the build will be marked unstable. >>

    So, in a Maven Jenkins job, if a test fails:

    • Maven build is SUCCESSFUL
    • Jenkins build is UNSTABLE

    Instead, in a freestyle Jenkins job executing Maven, if a test fails:

    • Maven build is FAILED
    • Jenkins build is FAILED

    Possible solutions:

    • Change the build to a freestyle Jenkins job running maven (which may be too much work though) or
    • Add the -Dmaven.test.failure.ignore=false option to your build (however, you would not have UNSTABLE builds any longer).