Search code examples
testngtestng-dataprovider

TestNG DataProvider returning object and RetryAnalyzer does not recognize retry


I've looked everywhere I know and I can't seem to find an answer for this one.

If I pass a "complex" object from DataProvider to test method the Invoker seems to get a new instance of RetryAnalyzer with each iteration.

IRL, my data provider makes a DB query and passes objects to the test method. I assumed originally the DB access was causing issues, but have since narrowed it down to what I think is simplest example.

Please see below. If I set a breakpoint in RetryAnalyzer I can see that retryCount is ALWAYS 0. Each time it is called it is a NEW instance. Test01 is basically an infinite loop.

TestNG 7.0.0

Any help?

Test class:

    package test;

    import org.testng.Assert;
    import org.testng.annotations.DataProvider;
    import org.testng.annotations.Test;

    public class RetryTester {

        public class DataObject extends Object {
            String name;
            String url;

            DataObject(String name, String url) {
                this.name = name;
                this.url = url;
            }
        }

        @Test(retryAnalyzer = RetryAnalyzer.class, dataProvider = "getData1")
        public void test01(DataObject d) throws Exception {
            System.out.println("[" + d.name + "]" + "/[" + d.url + "]");
            Assert.fail("Retry test fail");
        }

        @Test(retryAnalyzer = RetryAnalyzer.class, enabled = true, groups = { "load" }, dataProvider = "getData2")
        public void test02(String name, String url) throws Exception {
            System.out.println("[" + name + "]" + "/[" + url + "]");
            Assert.fail("Retry test fail");
        }

        @DataProvider
        public Object[][] getData1() {

            DataObject one = new DataObject("name1", "a");
            DataObject two = new DataObject("name2", "b");

            Object[][] data = new Object[][] { { one }, { two }, };

            return data;
        }

        @DataProvider
        public Object[][] getData2() {
            return new Object[][] { new Object[] { "name1", "a" }, new Object[] { "name2", "b" }, };
        }

    }

RetryAnalyzer class:

    package test;


    import org.testng.IRetryAnalyzer;
    import org.testng.ITestResult;

    public class RetryAnalyzer implements IRetryAnalyzer {
        private int retryCount = 0;
        private int retryMaxCount = 1; // set your count to re-run test

        @Override
        public synchronized boolean retry(ITestResult result) {
            if (retryCount < retryMaxCount) {
                System.out.println("Retry logic for  '" + result.getName() + "' on class " + this.getClass().getName());
                retryCount++;
                return true;
            }
            return false;
        }
    }

Test01 output:

    [name1]/[a]
    Retry logic for  'test01' on class test.RetryAnalyzer
    [name1]/[a]
    Retry logic for  'test01' on class test.RetryAnalyzer
    [name1]/[a]
    Retry logic for  'test01' on class test.RetryAnalyzer
    [name1]/[a]
    Retry logic for  'test01' on class test.RetryAnalyzer
    [name1]/[a]
    Retry logic for  'test01' on class test.RetryAnalyzer
    [name1]/[a]
    Retry logic for  'test01' on class test.RetryAnalyzer
    ...

Test02 output:

    [name1]/[a]
    Retry logic for  'test02' on class test.RetryAnalyzer
    [name1]/[a]
    [name2]/[b]
    Retry logic for  'test02' on class test.RetryAnalyzer
    [name2]/[b]
    FAILED: test02("name1", "a")
    FAILED: test02("name2", "b")
    RETRIED: test02("name1", "a")
    RETRIED: test02("name2", "b")

Solution

  • In Latest version '7.1.0' Its fixed

    PR:: Prevent Retry from happening endlessly #2165

    Fixed: GITHUB-2163: Test is executed infinite number of times when the data provider returns a new object (Krishnan Mahadevan)