Search code examples
testngjunit4

Unit-tests exhibiting 'state' across test-methods after moving from JUnit4 to TestNG


The legacy project I'm working on uses both JUnit4 and TestNG testing frameworks and for reasons beyond my control I cannot upgrade to JUnit5.

While all new test-suites (classes) that we create are written using TestNG, many old test classes still run on JUnit4. Primarily the difficulty in writing parameterized tests with JUnit4 (whether the conventional way or through JUnitParams extension) have been pushing us in direction of TestNG.


One test class that I'm adding more test-methods in is working with JUnit4. To have parameterized tests in it, I've moved from JUnit4 to TestNG (by changing import statements, updating annotations like @Before -> @BeforeClass etc.). However since then several tests have started failing with assertion errors.

  • Since its internal libraries I can't share the code-snippet here but basically the tests have started exhibiting a 'state' across different test-methods as a result of which the metrics we publish from our business logic are getting 'summed-up' across all different test methods.
  • Hence if I run each test method individually it passes, while running them in batch results in few tests passing with most others failing due to assertion-errors around metric counter values.

What could be wrong and how could I go about investigating it?


Solution

  • Thanks to @SyedSaubaan for guidance


    Turns out I had mixed up TestNG's @BeforeClass and @BeforeMethod annotations.

    @BeforeMethod: This will be executed before every @Test annotated method.

    @BeforeClass: This will be executed before first @Test method execution. It will be executed one only time throughout the test case.


    • After correctly relocating the metrics object initialization from @BeforeClass method to @BeforeMethod method, the assertions got fixed.
    • This of course happens because metrics object needs to be reset -> re-initialized just before running each test-method, so logically speaking it should be put under @BeforeMethod instead of @BeforeClass