JUnit - Looping test cases and comparing the results

Question

I am learning JUnit and I have to test a method multiple times. I have to test the method based on two parameters (Environment and Case#). I am working on two environments where I have to check if the same Case# yields the same results between different environments. This is the test case:

public class AppStartTest
{

/**
 * Test method for {@link archive.AppStart#beginOper(java.lang.String)}.
 */

List<String> actualSections = new ArrayList<String>();
List<String> environments = new ArrayList<String>();
List<String> cases = new ArrayList<String>();

@Before
public void prepareTest()
{
    environments.add("env1");
    environments.add("env2");
    cases.add("case1");
    //cases.add("case2");
    cases.add("case3");
    cases.add("case32");
    cases.add("case4");
    cases.add("emp3");

}

@Test
public void testBeginOper()
{
    for (String caseStr : cases)
    {
        @SuppressWarnings("unchecked")
        Map<String, Integer> mapList[] = new HashMap[2];
        int i = 0;
        for (String env : environments)
        {
            System.out.println("Starting " + env + "-" + caseStr);
            AppStart a = new AppStart();
            mapList[i] = a.beginOper(env + "-" + caseStr, true);
            printMap(mapList[i++]);             
        }
                    //Using assert in this method
        compareResults(mapList[0], mapList[1], caseStr);
    }
}   
}

The result yields as a single test case, but I would be requiring the results as:

testBeginOper[0]
testBeginOper[1]
testBeginOper[2]
.....

I tried using parameterized test cases, but the test method would be executed independently. I have to compare the results between two environments and I would need the method with @Test to return some value (Method should be void) and then assert. Please advise.

Answer 1

The easiest solution I can think of would be to not have testBeginOper() be annotated with @Test at all but just be a standard, non-test, value returning, parameter accepting function. Then you have one @Test annotated function which runs your various versions of testBeginOper(), collects the results and then compares them. It could look something like this:

private Map<String, Integer> testBeginOper(String environment, String case) {
    // do your actual tests using the given parameters "environment" and "case"
}

@Test
public void runTests() {
    List<Map<String, Integer>> results = new ArrayList<>();
    for(String environment : environments) {
        for(String case : cases) {
            results.add(testBeginOper(environment, case));
        }
    }
    // now compare your results
}

There are alternatives of course, one being to write your own TestRunner, but this is probably by far the easiest solution.