Search code examples
azure-devopsazure-pipelinesxunitvstest

Test methods are not executed when associating same test method to multiple test cases in VSTest running XUnit tests

We have an ASP.Net application and UI tests written with xUnit. Test plans are in VSTS, and in some cases same xUnit test method is associated to multiple test cases.

An azure build pipeline executes these tests using VSTest. The problem is when multiple test cases are associated with a single method seems only one of them is executed. E.g. The test cases in below screen shot are associated to same method and only one is executed.

enter image description here

We tried both 'Test assemblies' & 'Test Plan' option in Vstest, but results are same.

enter image description here

As per below link, it is not possible in xUnit to run the same test method multiple times in the same test session.

https://developercommunity.visualstudio.com/content/problem/995269/executing-multiple-test-cases-from-testplan-which.html?childToView=995554#comment-995554

Some solutions I can think of are,

  1. Creating dummy test methods for all test cases and maintain one to one, test method to test case mapping. Where one method will have testing logic, while other methods will just assert true.
  2. Create multiple test methods, where only one method will contain the implementation. Other methods will just call the test method which contains the implementation.

Please suggest if there is any better solution to the problem.

Thanks in advance!

Solution

  • Test methods are not executed when associating same test method to multiple test cases in VSTest running XUnit tests

    As we know:

    By default, each test class is a unique test collection. Tests within the same test class will not run in parallel against each other.

    So, the response we got from Azure Devops Developer Community forum and xunit is XUnit does not allow running one test multiple times in the same session.

    I personally think that the two workrounds you proposed are correct. You can use one of the two methods to solve this problem. You are already on the right way.

    Hope this helps.