Search code examples
azure-devopsazure-pipelinescicdazure-pipelines-release-pipelinevstest

How to rerun failed tests right after they fail in Azure DevOps release pipeline


I have a Azure DevOps release pipeline, which has VSTest@2 task configured to execute my selenium tests. It reruns the failed tests but it waits till the end of the execution of all the tests and then re-executes the failed tests. I have written some sequential tests which needs to be run one after another as the test data created by one test will be used by subsequent tests. I want the failed tests to be rerun immediately after it fails instead of being re-executed at the end. For example, if I have 5 tests and test#2 fails, I don't want the rest of tests to be executed till the failed test has been rerun and passed or the max rerun attempt has been reached. Is there any way to acheive this. Tech used: Selenium with C#, nUnit

The task is already configured to rerun failed tests but it does so after the execution of all the tests is complete. Task yaml variables: Category: 'Regression'

steps:

  • task: VSTest@2 displayName: 'VsTest - testAssemblies' inputs: testAssemblyVer2: '**/Automation.dll' testFiltercriteria: 'Category=$(Category)' failOnMinTestsNotRun: false rerunFailedTests: true continueOnError: true

Solution

  • Update on this: nunit has a "Retry" attribute, where you can give no. of retries you want. If there is an assertion failure in the test, the test execution will be retried at that instant and the execution won't wait for all the tests to get executed once.