How do I get the result of a Cucumber feature

Question

I'm trying to run Cucumber features in JUnit 5 Jupiter. I've lifted some code from the Cucumber-jvm source and adapted it for JUnit 5's TestFactory. It is working: I see my features running when I run all JUnit tests (this is Kotlin code, but the same applies to Java):

@CucumberOptions(
        plugin = arrayOf("pretty"),
        features = arrayOf("classpath:features")
)
class Behaviours {
    @TestFactory
    fun loadCucumberTests() : Collection<DynamicTest> {
        val options = RuntimeOptionsFactory(Behaviours::class.java).create()
        val classLoader = Behaviours::class.java.classLoader
        val resourceLoader = MultiLoader(classLoader)
        val classFinder = ResourceLoaderClassFinder(resourceLoader, classLoader)
        val runtime = Runtime(resourceLoader, classFinder, classLoader, options)
        val cucumberFeatures = options.cucumberFeatures(resourceLoader)
        return cucumberFeatures.map<CucumberFeature, DynamicTest> { feature ->
            dynamicTest(feature.gherkinFeature.name) {
                var reporter = options.reporter(classLoader)
                feature.run(options.formatter(classLoader), reporter, runtime)
            }
        }
    }
}

However, JUnit reports that every feature was successful, whether or not it actually was. When features fail, the results are correctly pretty-printed, but the generated DynamicTest passes. Neither gradle test nor Intellij notice the error: I have to inspect the text output.

I think I have to figure out, in the Executable passed as the second parameter to dynamicTest, what the result of the feature was, and raise an assertion when appropriate. How do I determine the result of feature or feature.gherkinFeature at that point?

And is there a way to get at the results for each scenario in the feature? Or better, is there a way to run a specific scenario, so that I can create a DynamicTest for each scenario, giving me better reporting granularity in JUnit?

Answer 1

To record the result of a Cucumber scenario as a JUnit5, I found it easiest to implement a JunitLambdaReporter, which is essentially a simpler version of the existing JunitReporter. Once you have a reporter that remembers what the current scenario is, then you can create a @TestFactory that uses this logic:

return dynamicTest(currentScenario.getName(), () -> {
  featureElement.run(formatter, reporter, runtime);
  Result result = reporter.getResult(currentScenario);

  // If the scenario is skipped, then the test is aborted (neither passes nor fails).
  Assumptions.assumeFalse(Result.SKIPPED == result);

  Throwable error = result.getError();
  if (error != null) {
    throw error;
  }
});