Search code examples
azure-devopsazure-pipelinesazure-pipelines-yamlazure-devops-pipelinesazure-pipelines-tasks

No tests found after splitting tasks out into separate jobs


I am trying to refactor my build pipeline by splitting some tasks out into separate jobs for extensibility.

The following works perfectly and the tests are found and ran without issue.

stages:
  - stage: build_test_publish
    displayName: Build
    pool:
      vmImage: "windows-latest"
    variables:
      solution: "**/SolutionName.sln"
      buildPlatform: "Any CPU"
      buildConfiguration: "Release"
    jobs:
      - job: build_publish_artifacts
        displayName: "Build, test and publish artifacts"
        steps:
          - task: NuGetToolInstaller@1
          - task: NuGetCommand@2
            inputs:
              command: "restore"
              restoreSolution: "**/*.sln"
              feedsToUse: "select"
              vstsFeed: "xxx"

          - task: VSBuild@1
            inputs:
              solution: "$(solution)"
              msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
              platform: "$(buildPlatform)"
              configuration: "$(buildConfiguration)"

          - task: VSTest@2
            inputs:
              platform: '$(buildPlatform)'
              configuration: '$(buildConfiguration)'
              runInParallel: false
              codeCoverageEnabled: true
              testAssemblyVer2: |
                 **\*test*.dll
                 !**\ProjectName.IntegrationTests.dll
                 !**\*TestAdapter.dll
                 !**\obj\**
              searchFolder: '$(System.DefaultWorkingDirectory)'

However after splitting it out like below I get the following output where no tests can be found.

stages:
  - stage: build_test_publish
    displayName: Build, test and publish artifacts
    pool:
      vmImage: "windows-latest"
    variables:
      solution: "**/SolutionName.sln"
      buildPlatform: "Any CPU"
      buildConfiguration: "Release"
    jobs: 
      - job: build
        displayName: "Build"
        steps:
          - task: NuGetToolInstaller@1
          - task: NuGetCommand@2
            inputs:
              command: "restore"
              restoreSolution: "**/*.sln"
              feedsToUse: "select"
              vstsFeed: "xxx"
          - task: VSBuild@1
            inputs:
              solution: "$(solution)"
              msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
              platform: "$(buildPlatform)"
              configuration: "$(buildConfiguration)"
                

      - job: test
        displayName: Run unit tests
        dependsOn: build
        steps:
          - task: VSTest@2
            inputs:
              platform: '$(buildPlatform)'
              configuration: '$(buildConfiguration)'
              runInParallel: false
              codeCoverageEnabled: true
              testAssemblyVer2: |
                 **\*test*.dll
                 !**\ProjectName.IntegrationTests.dll
                 !**\*TestAdapter.dll
                 !**\obj\**
              searchFolder: '$(System.DefaultWorkingDirectory)'
Starting: VSTest
==============================================================================
Task         : Visual Studio Test
Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version      : 2.170.1
Author       : Microsoft Corporation
Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : null
Search folder : D:\a\1\s
Action when minimum tests threshold not met : donothing
Minimum tests expected to be run: 0
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation with version [16.0,17.0).
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : true
Diagnostics enabled : false
SystemVssConnection exists true
Run the tests locally using vstest.console.exe
========================================================
Source filter: **\*test*.dll,!**\ProjectName.IntegrationTests.dll,!**\*TestAdapter.dll,!**\obj\**
##[warning]No test sources found matching the given filter '**\*test*.dll,!**\ProjectName.IntegrationTests.dll,!**\*TestAdapter.dll,!**\obj\**'
Finishing: VSTest

Why does this happen and what should I change or rethink?

**Edit Seems like it might be because each jobs uses it's own agent to run the tasks in it's job.

https://learn.microsoft.com/en-us/azure/devops/pipelines/get-started/key-pipelines-concepts?view=azure-devops

I would still like to know if something like this is possible in order to start test runs with different environment variables


Solution

  • DLL's produced by

              - task: VSBuild@1
                inputs:
                  solution: "$(solution)"
                  msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
                  platform: "$(buildPlatform)"
                  configuration: "$(buildConfiguration)"
    

    are not shared between jobs out of the box. Behind the scene splitting task you run VSBuild and VSTask on different machines. There is no much sense of doing that - I mean splitting these two tasks.

    However, if you still want to do this you need to publish artifact and download it in next job.