Search code examples
javaandroidgradlecucumber

How can I pass tags when starting a cucumber android run using Gradle?

I have a suite of cucumber android tests (they are instrumented tests) and I run them using gradlew connectedCheck. What I'm trying to do is run only certain tags when I run my tests (that's what tags are for, right?) using command-line options, without having to modify the code every time between test runs (so I can easily run these from CI server etc).

For example, I have tags @one and @two. I want to perform two gradle builds, the first with all the @one tests, and the second with all the @two tests. This would be part of an automated pipeline, so I wouldn't be able to modify the code (with the @CucumberOptions) between these two builds.

Here's what I've tried so far:

Attempt 1: Multiple CucumberOptions annotated classes

I tried having multiple runners (subclasses of CucumberAndroidJUnitRunner) and a different @CucumberOptions on each one, specifying different tags, then controlling which one was used via Gradle properies. I then did two builds, one with each runner. However, in both cases, Cucumber just used the @CucumberOptions annotation from the first runner alphabetically, so the tags were the same on both test runs. (In other words, even if the runner in use has a @CucumberOptions annotation, that's not necessarily the one that gets used.)

Attempt 2: Passing system property using gradle -D option

I found a hint from this thread that I might be able to pass -Dcucumber.options="--tags @two" but I tried this and it didn't work. It still just took the tags from the @CucumberOptions and ignored whatever I passed on the command-line. (This thread was confusing anyway as it started off talking about Gradle but later it was talking about Maven. I am using Gradle rather than Maven obviously).

I also tried -Dcucumber.filter.tags="@two" (which I found from the cucumber-jvm docs, talking about Maven not Gradle) but this also didn't work, the same as above.

Other investigations

  • I looked at loads of online guides, but they were all based on either modifying the @CucumberOptions or running cucumber from the command-line (rather than via gradle)
  • Obviously Java doesn't let you define annotation values by method calls, even if they are static methods, so the value in @CucumberOptions has to be constant (I double checked this)
  • I could dynamically generate the @CucumberOptions-annotated class at runtime, but I couldn't figure out how to ensure that this happened before the cucumber instrumentation runner looked for it

Versions used

androidTestImplementation 'io.cucumber:cucumber-android:4.8.4'
androidTestImplementation 'io.cucumber:cucumber-picocontainer:4.8.1'
Solution

  • You can pass options through gradle using:

    -Pandroid.testInstrumentationRunnerArguments.key=value

    For example:

    -Pandroid.testInstrumentationRunnerArguments.tags=@two

    These options are then passed to instrumentation and handled by cucumber, see https://github.com/cucumber/cucumber-jvm/pull/597

    So you can use for example:

    gradlew connectedCheck -Pandroid.testInstrumentationRunnerArguments.tags=@two