"no runnable methods" on junit with custom annotation and all tests filtered

Question

I'm using JUnit 4.11, extended the BlockJUnit4ClassRunner and used a custom annotation. If the annotation is present with a certain parameter, the test will not be run.

This works fine, except for the case, that all tests in a class are filtered out but the custom test runner.

Basically exactly as described in this post: JUnit4 skip test(s) according to custom java annotations

I tried the workarounds mentioned in this post: JUnit: how to avoid "no runnable methods" in test utils classes

If the I introduce a dummy test, so that there is at least one test, that will always run, everything is fine. If all tests are not run, as they all have the annotation, I will get the "No runnable methods" exception.

I can live with the workaround, but there should be a better way. I'm thankful for any pointers or explanations what I did wrong or how to improve this situation.

Answer 1

You get this error due to the validation performed in validateInstanceMethods since computeTestMethods() returns an empty List:

protected void validateInstanceMethods(List<Throwable> errors) {
        validatePublicVoidNoArgMethods(After.class, false, errors);
        validatePublicVoidNoArgMethods(Before.class, false, errors);
        validateTestMethods(errors);

        if (computeTestMethods().size() == 0)
            errors.add(new Exception("No runnable methods"));
    }

You can override this method and remove the size validation or add your own validation which will take into consideration the annotated test methods.
Notice that this method is @Deprecated and will be removed in future versions.