Search code examples
c#azureconcurrencyparallel-processingxunit

Azure pipeline concurrent (not parallel) tests execution for xunit tests


I am using azure pipeline to execute API tests, the tests are executed serially and I have hit the point where my job runs for over 1 hour - this means agent fails because 1 hour is the maximum job execution time. I have started to read how to execute tests in parallel, and in xunit tests should run in parallel by default when they are not in the same collection. In azure however, following https://learn.microsoft.com/en-us/azure/devops/pipelines/test/parallel-testing-vstest?view=azure-devops, to run tests parallelly it is required to use more than one agent and apparently more than 1 job. There are two cons in my case:

  • we have limited amount of agents and other people want to use them too, I do not want to occupy more agents than necessary;
  • I need to configure two firewall for cosmosdb for each job (1 job = 1 agent and each agent has new ip i need to add to firewall), this tasks takes 8 minutes and i need to configure 2 databases (16 minutes), each new job would mean i need to speend additional 16 minutes configuring firewalls.

So my question is - is it possible to run tests concurrent rather than parallel by utilizing same core? I have many thread.sleeps (but in my opinion i use them the correct way as i wait for some result for some time like 60 seconds, but i thread sleep for 2 seconds and every 2 seconds i'm polling in example databse if results are there) - thread.sleep means thread is sleeping and core should be available for other threads to run.


Solution

  • Apparently I have posted question too soon, found out that all I need to do is to set [assembly: CollectionBehavior(MaxParallelThreads = n)] to number I want (source: https://xunit.net/docs/running-tests-in-parallel). Default is number of logical processor and in case of azure agents this number is equal to 1.