I have a wait test that I expect to fail on occasion because the test threads get too out of sync. So I'm setting the successPercentage and invocationCount to handle the occasional acceptable failure. If the code's broken, none of the invocations will pass.
When I run the test in IntelliJ, everything's fine. But when I try and do a build in maven, I get the following
Tests run: 1407, Failures: 0, Errors: 1, Skipped: 0 [INFO] ------------------------------------------------------------------------ [ERROR] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] There are test failures.
Looking in restng-results.xml, I see this:
<test-method status="SUCCESS_PERCENTAGE_FAILURE" signature="waits()" name="waits" duration-ms="1000" started-at="2012-03-02T13:51:58Z" finished-at="2012-03-02T13:51:59Z">
And in TEST-TestSuite.xml, I see:
<testcase classname="com.casenet.integration.filesystem.TriggerFileListenerTest" time="1" name="waits">
<error type="java.lang.AssertionError" message="expected:<EXECUTE> but was:<TIMED_OUT>">java.lang.AssertionError: expected:<EXECUTE> but was:<TIMED_OUT>
The rest of the testng generated files seem to think everything passed. But it looks like that surefire is getting confused by SUCCESS_PERCENTAGE_FAILURE or the <error/> element into thinking something's gone wrong when in fact nothing unexpected happened.
Anyone know if this is a known problem or if there's a solution available?
Looks like there is an open surefire bug on this, which also seems to have a patch and workaround.