Search code examples
javamaventestingtestng

Why custom listener is called when I don't attach it to TestClass, not via annotation nor via the suite xml file? (TestNG)


Wondering why a customListener of mine is getting called even If I don't attach it to TestClass and not to xml file..?

So I've made a custom logger listener that implements ITestListener, IClassListener.
Now I attached this listener to 1 of the of my tests classes, and the second test class doesn't have a listener attached to it.
Now I've put both of the tests classes into, 1 suite xml file.

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite A">
    <test name="FirstSuiteFirstClassTest" >
        <classes>
            <class name="ITClassTestA"/>
        </classes>
    </test>
    <test name="FirstSuiteSecondClassTest" >
        <classes>
            <class name="ITClassTestB"/>
        </classes>
    </test>
</suite>
@Listeners({LoggerListener.class})
public class ITClassTestA extends BaseTest {

    @Test
    public void test1InClassTestA() throws InterruptedException {
        logger.info("Started test 1 in ClassTest A: " + TimeUtils.nowUTC());
        Thread.sleep(TimeUnit.SECONDS.toMillis(4));
        logTheName("SDK");
        logger.info("Ended test 1 in ClassTest A: " + TimeUtils.nowUTC());
    }

    @Test
    public void test2InClassTestA() throws InterruptedException {
        logger.info("Started test 2 in ClassTest A: " + TimeUtils.nowUTC());

        Thread.sleep(TimeUnit.SECONDS.toMillis(4));

        logger.info("Ended test 2 in ClassTest A: " + TimeUtils.nowUTC());
    }
}
public class ITClassTestB extends BaseTest {

    @Test
    public void test1InClassTestB() throws InterruptedException {
        logger.info("Started test 1 in ClassTest B at: " + TimeUtils.nowUTC());
        Thread.sleep(TimeUnit.SECONDS.toMillis(4));
        logTheName("testNG");
        logger.info("Ended test 1 In ClassTest B at: " + TimeUtils.nowUTC());
    }

    @Test
    public void test2InClassTestB() throws InterruptedException {
        logger.info("Started test 2 In ClassTest B at: " + TimeUtils.nowUTC());

        Thread.sleep(TimeUnit.SECONDS.toMillis(4));

        logger.info("Ended test 2 In ClassTest B at: " + TimeUtils.nowUTC());
    }
}
public abstract class BaseTest {
    protected Logger logger;

    @BeforeMethod
    public void createLog() {

    }

    @BeforeClass
    public void beforeAll() {
        if (logger == null) {
            logger = LogManager.getLogger(this.getClass().getSimpleName());
        }
    }

    @AfterClass
    public void afterAll() {
        logger.info("Bye dude");
    }


    protected void logTheName(String name) {
        logger.info("The name is {}", name);
    }
}
public class LoggerListener implements ITestListener, IClassListener {
    private final Map<String, List<TestResultStatus>> testResultsStatusPerClass = new ConcurrentHashMap<>();

    private enum TestResultStatus {
        SUCCESSFUL, FAILED, TIMED_OUT, SKIPPED;
    }

    @Override
    public void onBeforeClass(ITestClass testClass) {
        Class<?> testRealClass = testClass.getRealClass();
        String testClassName = testRealClass.getSimpleName();

        initLogFile(testClassName);
    }

    @Override
    public void onAfterClass(ITestClass testClass) {
        List<TestResultStatus> resultStatuses = testResultsStatusPerClass.get(testClass.getRealClass().getSimpleName());

        Map<TestResultStatus, Long> summary = resultStatuses
                .stream()

         `Here I get NPE, because resultStatuses doesn't find the test class name in the map`
                

I didn't attach the rest of the code of `LoggerListener`..
But the main thing I try to understand is how come onBeforeClass and onAfterClass are getting trigger while I didn't attach the logger listener to TestClassB.. I've seen that there is invokeBeforeClassMethods of testNg that is called in run time, and there is listener.onBeforeClass()... Couldn't get a conclusion.


Solution

  • TestNG provides the following 3 ways to wire in listeners into it.

    • via the @Listeners annotation
    • via the xml tag <listeners> in the suite file.
    • via service loaders approach.

    But irrespective of what mechanism was used to wire in a listener, listeners in TestNG are always GLOBAL in nature.

    So this means that the listener will get executed for all classes.

    If you would like your listener to selectively work for ONLY those classes on which it was defined, then you would need to ensure that you add that filtering logic in your test code.

    For e.g., you could alter your listener to look like below:

    import org.testng.IInvokedMethod;
    import org.testng.IInvokedMethodListener;
    import org.testng.ITestResult;
    import org.testng.annotations.Listeners;
    
    public class SampleLoggingListener implements IInvokedMethodListener {
    
        @Override
        public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
            boolean isPresent = testResult.getTestClass().getRealClass()
                    .isAnnotationPresent(Listeners.class);
            if (isPresent) {
                System.err.println("Executing method " + method.getTestMethod().getQualifiedName());
            }
        }
    }
    

    Update: Listener variant that enforces 1:1 mapping between a listener and the test class it was meant for.

    This below listener will ensure that it gets executed ONLY when:

    • The test class has the annotation @Listeners
    • The @Listeners annotation should contain the current listener's name as one of the listeners
    import org.testng.IInvokedMethod;
    import org.testng.IInvokedMethodListener;
    import org.testng.ITestResult;
    import org.testng.annotations.Listeners;
    
    import java.util.Arrays;
    
    public class SampleLoggingListener implements IInvokedMethodListener {
    
        @Override
        public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
            Class<?> klass = testResult.getTestClass().getRealClass();
            Listeners listeners = klass.getAnnotation(Listeners.class);
            if (listeners == null) {
                return;
            }
            //Check if the test class was:
            //1. Annotated using @Listeners annotation
            //2. If the current listener is one of the listeners specified in that @Listeners annotation
            boolean isPresent = Arrays.asList(klass.getAnnotation(Listeners.class).value())
                    .contains(getClass());
            if (isPresent) {
                System.err.println("Executing method " + method.getTestMethod().getQualifiedName());
            }
        }
    }