Search code examples
azureazure-devopsintegration-testing

Azure DevOps Releases - How to carry out a Test Stage at a specific virtual machine?

After Azure => Pipelines, I end up with two published artifacts: one containing a .NET Core Console Application (myDrop), another containing the corresponding Testing library (written with xUnit) (myTestDrop). Then I move on to Azure => Releases to create a new Release Pipeline as below:

enter image description here

I have a Windows Virtual Machine (VM), which is already installed with all necessary libraries, e.g. .NET Core; and I would like to carry out the Integration Testing (the 2nd Stage above) in that machine. Specifically,

  1. Copy both myDrop and myTestDrop to that VM.
  2. Set some Environment Variable: the path to, let's say, MyConsole.exe in myDrop.
  3. Then run the Integration Test: dotnet vstest "MyConsole.Tests.dll" --logger:trx --ResultsDirectory:"c:/Somewhere" /TestCaseFilter:"Category=IntegrationTest"
  4. If the test is successful, the returned code from dotnet.exe is 0 (otherwise, 1).
  5. The 3rd Stage only runs in case the 2nd Stage is successful.
  6. There should be a way to read *.trx generated from the Integration Test above, especially in case there are some test failures.

My experience with Azure DevOps is limited. I have searched around but most Azure Release examples involve Web Application (IIS, SQL...), not a normal Console Application with Test on a specific VM. Feel free to suggest other alternatives or best practices, given the scenario above.

Any advice, suggestions are appreciated.

Solution

  • How to carry out a Test Stage at a specific virtual machine?

    You can install and use a self-hosted agent in that machine. Please refer to this document to install Self-hosted Windows agents. You need to add an agent job to the release pipeline first. Then, choose the agent pool with self-hosted agent installed. enter image description here

    Copy both myDrop and myTestDrop to that VM.

    Since your agent is installed on the VM, it will automatically download artifacts to its local folder.

    The 3rd Stage only runs in case the 2nd Stage is successful.

    You can select “After Stage” trigger in pre-deployment conditions. For example, if the test stage in your screenshot fails, the deployment stage will not deploy. enter image description here

    There should be a way to read *.trx generated from the Integration Test above, especially in case there are some test failures.

    You can check the test results in the Tests tab of the release result page. You can also download the *.trx file on this page. enter image description here