Search code examples
javajenkinstestngcucumber-java

How to change Jenkins build status if failed scenarios were successfully reruned


I have two TestNG runners to run main and failed scopes of cucumber scenarios. In case MainTestRunner was successfull, Jenkins build status is "success" and FailedTestRunner will not find any failed scenarios into rerun.txt. But in case MainTestRunner has failed scenario and it was reruned in scope of FailedTestRunner and passed, Jenkins build is failed anyway.

I need a solution to make build status "green" as all tests are passed in the end.

There is no special tricks in my test runners:

MainTestRunner:

@CucumberOptions(
        features = ".",
        glue = {"steps"},
        monochrome = true,
        format = {
                "pretty",
                "html:target/cucumber-pretty",
                "json:target/CucumberTestReport.json",
                "rerun:target/rerun.txt"
        })

public class MainTestRunner extends AbstractTestNGCucumberTests {
private TestNGCucumberRunner testNGCucumberRunner;

@BeforeClass(alwaysRun = true)
public void setUpClass() {
    testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
}

@Test(description = "Runs Cucumber Feature", dataProvider = "features")
public void feature(CucumberFeatureWrapper cucumberFeature) {
    testNGCucumberRunner.runCucumber(cucumberFeature.getCucumberFeature());
}

@DataProvider
public Object[][] features() {
    return testNGCucumberRunner.provideFeatures();
}

@AfterClass(alwaysRun = true)
public void tearDownClass() {
    testNGCucumberRunner.finish();
}
}

FailedTestRunner:

@CucumberOptions(
    features = "@target/rerun.txt"
    , glue = {"steps"}
    , monochrome = true
    , format = {
    "pretty",
    "html:target/cucumber-pretty",
    "json:target/CucumberTestReport.json",
    "rerun:target/rerun.txt"
})

public class FailedTestRunner extends AbstractTestNGCucumberTests {}

Solution

  • The main goal was to get 'green' Jenkins status if second test run was successful.
    I added one more build step. Now I'm using:

    mvn clean test -Dmaven.test.failure.ignore=true
    

    to record all failed tests to rerun.txt file.
    And:

    mvn test -Dtest=runners.FailedTestRunner -DfailIfNoTests=false
    

    to run it.