I am looking to maximize the parallel execution of Specflow + Xunit, currently the parallel is up to the feature level. I am looking for a way to make it up to scenario level. I do plan to use specflow+runner, unfortunately it has stopped development.
Currently I did the workaround using DevOps pipeline to distribute the scenarios to 2 agents. Does someone have any better suggestions?
Calculator.feature
Feature: Calculator
Scenario: Add two numbers
...
@agent2 // Assign to agent 2
Scenario: Add two numbers 2
...
Calculator2.feature
Feature: Calculator2
Scenario: Add two numbers 3
...
@agent2 // Assign to agent 2
Scenario: Add two numbers 4
...
This appears to be a limitation of xUnit. From Running Tests in Parallel on xunit.net:
Test Collections
How does xUnit.net decide which tests can run against each other in parallel? It uses a concept called test collections to make that decision.
By default, each test class is a unique test collection. Tests within the same test class will not run in parallel against each other.
(emphasis, mine)
A feature file in SpecFlow gets parsed and converted to a test class in xUnit. A test class (and therefore a feature file in SpecFlow) become default collections to support parallelization when running tests. Until xUnit supports a way to run test methods in the same class in parallel, this does not appear possible when using xUnit + SpecFlow, because it is not possible when just using xUnit.