Rerun failing tests with different log4j config (set to debug)?

Question

We have a multi-module maven project where each module contains tests. We have about 2000 tests, from unit tests to integration tests.

We use TestNG and Log4j

These integration tests do quite some work. During development, we sometimes add a System.out.println to print some debugging information.

I know syso is generally not a good idea, so I try to use Log4j.

Is there an easy way to automatically re-run failed tests with Maven and TestNG with an different log4j configuration

Answer 1

I came up with a solution (based on the comment above)

• A setting to set the global Log4j config to ERROR
• A setting to change it to DEBUG on the 1st retry, and reset to ERROR after
• A transformer so I don't need to change the annontation for each @Test

I implemented three classes:

Usage

Just run: mvn test

Relevant part of pom.xml

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven.surefire.plugin.version}</version>
            <configuration>
                <suiteXmlFiles>
                    <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
                </suiteXmlFiles>
            </configuration>
        </plugin>

(To make this work in maven surefire you also need to add these classes to the testng.xml)

testng.xml

<suite name="Suite1" verbose="1">
    <listeners>
        <listener class-name="my.package.RerunBeforeSuiteOnlyError" />
        <listener class-name="my.package.RerunTestTransformer" />
    </listeners>
    ...

RerunBeforeSuiteOnlyError

public class RerunBeforeSuiteOnlyError implements ISuiteListener {

    public static final Level DEFAULT_TEST_LOGGING_LEVEL = Level.ERROR;

    @Override
    public void onStart(final ISuite suite) {
        Configurator.setRootLevel(DEFAULT_TEST_LOGGING_LEVEL);
    }

    @Override
    public void onFinish(final ISuite suite) {

    }
}

RerunDebugModifier

public class RerunDebugModifier implements IRetryAnalyzer {

    boolean retried = false;

    @Override
    public boolean retry(final ITestResult result) {

        // do we need to retry?
        // 1st time YES -> retry with DEBUG level
        if (!this.retried) {
            Configurator.setRootLevel(Level.DEBUG);
            this.retried = true;
            return true;

        } else {
            // 2nd time NO -> reset to ERROR, don't retry
            Configurator.setRootLevel(RerunBeforeSuiteOnlyError.DEFAULT_TEST_LOGGING_LEVEL);

            return false;
        }

    }
}

RerunTestTransformer

public class RerunTestTransformer implements IAnnotationTransformer {


    @Override
    public void transform(final ITestAnnotation testannotation, final Class testClass, final Constructor testConstructor, final Method testMethod) {

        final IRetryAnalyzer retry = testannotation.getRetryAnalyzer();

        if (retry == null) {
            testannotation.setRetryAnalyzer(RerunDebugModifier.class);
        }
    }
}