Search code examples
javaunit-testingjunittest-suitetest-runner

Failures are not collected in dynamic created test suite


I have a jar file that I reflect from it test classes (junit test).

I created TestSuite instance and added the tests to it.

In order to check my code, I've tried to add only one example test class to the test suite.

TestSuite suite = new TestSuite();
suite.addTest(new JUnit4TestAdapter(ExampleTest.class));

Then I called suite.run() in order to run the test:

TestResult result = new TestResult();
suite.run(result);

The problem is that when the test is done, the failures list that should be in result is empty.

Enumeration<TestFailure> failures = result.failures(); // = empty list

How can I get the failures using testSuite? If I use JUnitCore.runclasses , I do get the failures list but it can't be used with an instance of TestSuite, which I have to use since I get the test classes as input.

Edit-1:

If it is possible to create a Class that extends TestSuite and add classes to the suite dynamically, it will be good for my needs to.
Any suggestion will be great.

Edit-2:

From more searching in the web, I saw there us a difference between a failure and a testFailure. How can I fail a test with testFailure and not a failure?


Solution

  • After some more debugging, I found out that the TestResult object did catch the failures but reported them as Errors (Assertion Errors).
    I also found out that TestResult.errors() returned an enumerator of TestFailure. Each TestFailure contain information about the thrown exception, so I can distinguish now between errors and failures by checking which of the errors are Assertion Errors and which are not.