Search code examples
c#azure-pipelinesmstest

Azure pipeline unit test Uri for TestContext.DeploymentDirectory causes failure


I have an Azure DevOps build that is executing a suite of unit tests. The tests pass locally but the build fails on the Hosted 2017 agent because of a failed test that is using TestContext.DeploymentDirectory.

The failing test creates a mock settings object using the TestContext. The OutputPath for the mock settings is set to the TestContext.DeploymentDirectory value. The mock settings object is then passed to a constructor and the resulting object checked with an Assert.AreEqual as shown;

[TestMethod]
public void Test_DiskStorageManager()
{
  var settings = new MockDataHandlerSettings(TestContext);
  var manager = new DiskStorageManager(settings);

  Assert.AreEqual(settings.OutputPath, manager.OutputPath);
}

Within the constructor is the following;

if (Uri.TryCreate(settings.OutputPath, UriKind.Absolute, out Uri outputUri))
{
  OutputPath = outputUri.LocalPath;
  IsLocalOutput = outputUri.IsLoopback;
}
else
{
  throw new ArgumentException($"Output path is invalid {settings.OutputPath}");
}

The tests run fine locally but the following test failure occurs on the hosted agent;

Assert.AreEqual failed. 
Expected:<D:\a_temp..\Tests\TestResults\TestResults\Deploy_VssAdministrator 2020-03-10 03_09_31\Out>. 
Actual:<D:\a\Tests\TestResults\TestResults\Deploy_VssAdministrator 2020-03-10 03_09_31\Out>.

What is causing the change from D:\a_temp..\ to D:\a\ and making the test fail?

EDIT

The testing task is defined by the following YAML;

variables:
  BuildConfiguration: 'debug'
  BuildPlatform: 'any cpu'

steps:
- task: VSTest@2
  displayName: 'Test Assemblies'
  inputs:
    testAssemblyVer2: |
     **\$(BuildConfiguration)\**\*Tests.dll
     !**\obj\**
    runSettingsFile: '$/MyProject/Tests/.runsettings'
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'

The .runsettings contains

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <RunConfiguration>
        <MaxCpuCount>1</MaxCpuCount>
        <ResultsDirectory>..\Tests\TestResults</ResultsDirectory>
        <TargetPlatform>x64</TargetPlatform>
    </RunConfiguration>
  <DeploymentEnabled>True</DeploymentEnabled>
</RunSettings>

Solution

  • The test fails because of the relative path in the ResultsDirectory element of the .runsettings file. The tests intialize with;

    TestResultsDirectory intialized to D:\a\_temp\..\Tests\TestResults\TestResults
    

    Removing <ResultsDirectory>..\Tests\TestResults</ResultsDirectory>;

    TestResultsDirectory intialized to d:\a\_temp\TestResults
    

    Note: The expected value displayed in the failed test has not correctly escaped the \. The displayed value is D:\a_temp..\Tests but the failed message in the task log is D:\a\_temp\..\Tests.

    This does not explain why the relative path is fully resolved in VS2019 16.4 and not in the Azure pipeline.