Exclusion by annotation not working for sbt test/testOnly

Question

Consider the following org.scalatest.TagAnnotation subclass:

public class TestSizeTags {

    /** Tests with crazy long runtimes **/
    @org.scalatest.TagAnnotation
    @Inherited
    @Retention(RetentionPolicy.RUNTIME)
    @Target({ElementType.METHOD, ElementType.TYPE})
    public static @interface HugeTestClass {}

}

Let us annotate/tag a class with it:

@HugeTestClass
class ItemsJobTest extends FunSuite with BeforeAndAfterEach with DataFrameSuiteBase {

Now we want a quick "smoke test suite" on the codebase; therefore, let us (attempt) to exclude the testcases annotated by HugeTestClass :

Command line:

sbt test * -- -l HugeTestClass

OR maybe:

sbt 'testOnly * -- -l HugeTestClass' 

Let us also attempt it within sbt itself:

sbt> testOnly * -- -l HugeTestClass

In all cases above we (unfortunately) still see:

[info] ItemsJobTest:
^C[info] - Run Items Pipeline *** FAILED *** (2 seconds, 796 milliseconds)

So the test actually did run.. contrary to the intention.

So what is the correct syntax / approach to apply a Tag Filter(/Exclusion) via sbt to scalatest classes?

Answer 1

You missed to put the testOnly part in double quote and also give full package to the Tag Annotation to ignore,

sbt "test-only * -- -l full.package.to.HugeTestClass"

example,

Tag annotation

package tags;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@org.scalatest.TagAnnotation
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface ExcludeMePleaseTag {}

Test to exclude

@tags.ExcludeMePleaseTag
class ExcludeMeSpecs extends FlatSpec with Matchers {

  "I" should " not run" in {

    888 shouldBe 1
  }

}

to exclude the tests

sbt "test-only * -- -l tags.ExcludeMePleaseTag"

This github issue was helpful - https://github.com/harrah/xsbt/issues/357#issuecomment-44867814

But it does not work with static Tag annotation,

public class WrapperClass {

    @org.scalatest.TagAnnotation
    @Retention(RetentionPolicy.RUNTIME)
    @Target({ElementType.METHOD, ElementType.TYPE})
    public static @interface ExcludeMePleaseTag {
    }

}

sbt "test-only * -- -l tags.WrapperClass.ExcludeMePleaseTag"