Search code examples
azure-devopsvisual-studio-test-runner

VSTS "Visual Studio Test" task deletes TRX file after publish


I've inherited a jacked VSTS build process and I'm attempting to integrate SonarQube into the mix. Jacked, in that it isn't a clean install and I know the previous owner has done some strange and mysterious things with it that I haven't been able to track down yet.

Anyways, the problem I'm having is that the "Visual Studio Test" task step is deleting the .trx file after publishing the results. I need the .trx file for SonarQube. I can see the trx file is being deleted and after a screen recording of the folder and the active console logging of the VSTS build process pinpointed the publish process where the .trx file disappears. With system.debug = true here are a subset of the logs (the .trx file disappears somewhere in here):

Test results remaining: 165. Test run id: 33.
Leaving UpdateTestResultsAsync
Updated test results: 165
Publishing Attachments: 2
Entering CreateTestRunAttachmentsAsync
Leaving CreateTestRunAttachmentsAsync
Uploading test run attachements individually
Completed PublishTestResult
Completed Publish Test Results: 0

I've tried a number of things, the latest of which was to install the latest VS 2017 Enterprise IDE to the build machine and am using these settings in the task: enter image description here

enter image description here

Not sure if this helps but here is an approximation of the vstest call the above makes as shown in the console:

vstest.console.exe /TestCaseFilter:"FullyQualifiedName=<all the dlls in my solution>"
"C:\agent\_work\2\s\Source\Sln\Project1\bin\Debug\net461\Project1.Tests.dll"
"C:\agent\_work\2\s\Source\Sln\Project2\bin\Debug\net461\Project2.Tests.dll"
"C:\agent\_work\2\s\Source\Sln\Project3\bin\Debug\net461\Project3.Tests.dll"
/EnableCodeCoverage
/InIsolation
/logger:"trx"
/TestAdapterPath:"C:\agent\_work\2\s\Source\Sln"
/diag:"C:\agent\_work\_temp\4acf64a0-4b51-11e8-9bf5-3b4a52af383b.txt"

-#############################

If I run vstest.console.exe from the command line, the results aren't published and the .trx file isn't deleted.

-#############################

I'd like the Visual Studio Test task step to publish but not delete the .trx file.

Any help would be appreciated. Thanks!

-###################

Update 1: Additional logging of the final "Completed Test Execution" section:

**************** Completed test execution *********************
Current Phase: MS.VS.TestService.VstestConsoleAdapter.ExecuteVsTestPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.ExecutionVsTestPhaseResults
PERF WARNING: Running the phase MS.VS.TestService.VstestConsoleAdapter.ExecuteVsTestPhase: took 5414.7701 ms
Current phase: MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhase
Starting Publish Test Results: TIA: False
Starting PublishTestResult
Test results files: C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27.trx
Updating Test results: 35
Entering ParseTestResultFiles
runContext.Platform is null
runContext.Configuration is null
runContext.ReleaseURI is null
runContext.ReleaseEnvironmentUri is null
Reading test results from file 'C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27.trx'.
Setting run start and finish times.
Attachment location: C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27\In.
Adding run level attachment: C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27\In\SERVERNAME\SYSTEM_SERVERNAME 2018-04-30 07_43_21.coverage.
Total test results: 168.
Leaving ParseTestResultFiles
pdating test results: 168
Entering UpdateTestResultsAsync
Publishing test results to test run '35'.
PERF: GetTestCaseResults: took 333.1965 ms
PERF WARNING: GetTestCaseResults: took 333.1965 ms
Test results remaining: 165. Test run id: 35.
Leaving UpdateTestResultsAsync
Updated test results: 165
Publishing Attachments: 2
Entering CreateTestRunAttachmentsAsync
Leaving CreateTestRunAttachmentsAsync
Uploading test run attachements individually
Completed PublishTestResult
Completed Publish Test Results: 0
Current Phase: MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhaseResults
PERF WARNING: Running the phase MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhase: took 1719.7748 ms
Current phase: MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhase
Started RerunValidationPhase for 35
Maximum number of attempts running the failed tests reached: 3
Processed: ##vso[task.logissue type=warning;]Maximum number of attempts running the failed tests reached: 3
Current Phase: MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhaseResults
PERF: Running the phase MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhase: took 1.1836 ms
Current phase: MS.VS.TestService.VstestConsoleAdapter.CleanupPhase
Started CleanupPhase 35
Test run needs to be marked as complete.
Entering UpdateTestRunAsync
Leaving UpdateTestRunAsync
Completed CleanupPhase 35
Current Phase: MS.VS.TestService.VstestConsoleAdapter.CleanupPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.CleanupPhaseResults
PERF: Running the phase MS.VS.TestService.VstestConsoleAdapter.CleanupPhase: took 259.2514 ms
Completed the ExecutionStateModel
Completed TestExecution Model...
rc:0
success:true
File exists. Size: 1224675 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.txt
Agent.Version=2.133.3
Agent.TempDirectory=C:\agent\_work\_temp
defaultRoot: 'C:\agent\_work\_temp'
findOptions.followSpecifiedSymbolicLink: 'true'
findOptions.followSymbolicLinks: 'true'
matchOptions.debug: 'false'
matchOptions.nobrace: 'true'
matchOptions.noglobstar: 'false'
matchOptions.dot: 'true'
matchOptions.noext: 'false'
matchOptions.nocase: 'true'
matchOptions.nonull: 'false'
matchOptions.matchBase: 'false'
matchOptions.nocomment: 'false'
matchOptions.nonegate: 'false'
matchOptions.flipNegate: 'false'
pattern: '*host.*.txt'
findPath: 'C:\agent\_work\_temp'
statOnly: 'false'
findPath: 'C:\agent\_work\_temp'
findOptions.followSpecifiedSymbolicLink: 'true'
findOptions.followSymbolicLinks: 'true'
C:\agent\_work\_temp (directory)
C:\agent\_work\_temp\.taskkey (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-43-19_00381_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-25_11343_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-35_20720_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-42_87908_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-43-22_36319_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-25_98843_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-36_03532_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-43_70721_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b061-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b063-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\proxy (file)
14 results
found 14 paths
applying include pattern
adjustedPattern: 'C:\agent\_work\_temp\*host.*.txt'
4 matches
pattern: '*datacollector.*.txt'
findPath: 'C:\agent\_work\_temp'
statOnly: 'false'
findPath: 'C:\agent\_work\_temp'
findOptions.followSpecifiedSymbolicLink: 'true'
findOptions.followSymbolicLinks: 'true'
C:\agent\_work\_temp (directory)
C:\agent\_work\_temp\.taskkey (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-43-19_00381_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-25_11343_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-35_20720_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-42_87908_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-43-22_36319_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-25_98843_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-36_03532_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-43_70721_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b061-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b063-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\proxy (file)
14 results
found 14 paths
applying include pattern
adjustedPattern: 'C:\agent\_work\_temp\*datacollector.*.txt'
4 matches
8 final results
File exists. Size: 11672 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-43-19_00381_1.txt
File exists. Size: 11672 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-25_11343_1.txt
File exists. Size: 11671 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-35_20720_1.txt
File exists. Size: 11582 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-42_87908_1.txt
File exists. Size: 413384 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-43-22_36319_1.txt
File exists. Size: 45500 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-25_98843_1.txt
File exists. Size: 45350 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-36_03532_1.txt
File exists. Size: 45500 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-43_70721_1.txt
task result: Succeeded
Processed: ##vso[task.complete result=Succeeded;]VsTest task succeeded.
Agent.Version=2.133.3
Release.ReleaseUri=undefined
Release.ReleaseId=undefined
Build.BuildUri=vstfs:///Build/Build/745
Build.Buildid=745
Agent.Version=2.133.3
telemetry area: TestExecution feature: TestExecutionTask data: {"builduri":"vstfs:///Build/Build/745","buildid":"745","codeCoverageEnabled":true,"overrideTestrunParameters":"false","pipeline":"build","runTestsInIsolation":true,"task":"VsTestConsoleFlow","runInParallel":false,"result":"Succeeded","settingsType":"none","testSelection":"testAssemblies","tiaEnabled":false,"vsTestVersion":"15.0.27428","consoleOptionsEnabled":"","rerunEnabled":true,"rerunType":"basedOnTestFailurePercentage"}
Processed: ##vso[telemetry.publish area=TestExecution;feature=TestExecutionTask;]{"builduri":"vstfs:///Build/Build/745","buildid":"745","codeCoverageEnabled":true,"overrideTestrunParameters":"false","pipeline":"build","runTestsInIsolation":true,"task":"VsTestConsoleFlow","runInParallel":false,"result":"Succeeded","settingsType":"none","testSelection":"testAssemblies","tiaEnabled":false,"vsTestVersion":"15.0.27428","consoleOptionsEnabled":"","rerunEnabled":true,"rerunType":"basedOnTestFailurePercentage"}

Solution

  • I can also reproduce the result.

    For VS Test task with the version 2.*, it will delete the .trx file after executing VS Test task.

    If you need to get the .trx file after VS Test task, you can use VS Test task with the version 1.*. It will keep the .trx file in the source directory.

    enter image description here

    enter image description here