unit-testing .net-core azure-devops azure-pipelines azure-pipelines-yaml

How to ensure Azure Pipelines accepts my trx files to publish test results?

I created a trx file in my pipeline for my integration tests created with Azure pipelines. It is working and and looks as expected.

<?xml version="1.0" encoding="utf-8"?>
<TestRun id="d5c0721d-bdc9-4900-a1bb-dcd38397a39a" name="VssAdministrator@fv-az520-777 2024-07-22 10:19:50" runUser="fv-az520-777\VssAdministrator" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2024-07-22T10:19:50.2469008+00:00" queuing="2024-07-22T10:19:50.2469017+00:00" start="2024-07-22T10:19:47.3501168+00:00" finish="2024-07-22T10:19:50.2550498+00:00" />
  <TestSettings name="default" id="f3d1a7c1-f83b-4d16-801a-b5e375be3fba">
    <Deployment runDeploymentRoot="VssAdministrator_fv-az520-777_2024-07-22_10_19_50" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="7e3e6863-f27c-4832-a2ca-aa0589a48439" testId="45ece44b-8380-88b4-513f-f71b031f1b40" testName="MyCompany.ConsumerApi.IntegrationTest.MyApproachContextTest.ProcessTest" computerName="fv-az520-777" duration="00:00:01.8936378" startTime="2024-07-22T10:19:50.1224192+00:00" endTime="2024-07-22T10:19:50.1224444+00:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="7e3e6863-f27c-4832-a2ca-aa0589a48439">
      <Output>
        <StdOut>2024-07-22 10:19:48 [Information] Application starting up...&#xD;
2024-07-22 10:19:48 [Information] KeyVault found False&#xD;
2024-07-22 10:19:50 [Information] Executed DbCommand (38ms) [Parameters=[], CommandType='Text', CommandTimeout='30']&#xD;
SELECT [b].[Id], [b].[FileMetadataId], [b].[Identifier]&#xD;
FROM [AttachmentSource] AS [b]</StdOut>
      </Output>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="MyCompany.ConsumerApi.IntegrationTest.MyApproachContextTest.ProcessTest" storage="d:\a\1\s\10 - MyApproach\c2woo.MyApproach\MyCompany.ConsumerApi.integrationtest\bin\release\net8.0\MyCompany.ConsumerApi.integrationtest.dll" id="45ece44b-8380-88b4-513f-f71b031f1b40">
      <Execution id="7e3e6863-f27c-4832-a2ca-aa0589a48439" />
      <TestMethod codeBase="D:\a\1\s\10 - MyApproach\c3wdd.MyApproach\MyCompany.ConsumerApi.IntegrationTest\bin\Release\net8.0\MyCompany.ConsumerApi.IntegrationTest.dll" adapterTypeName="executor://xunit/VsTestRunner2/netcoreapp" className="MyCompany.ConsumerApi.IntegrationTest.MyApproachContextTest" name="ProcessTest" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="45ece44b-8380-88b4-513f-f71b031f1b40" executionId="7e3e6863-f27c-4832-a2ca-aa0589a48439" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="1" executed="1" passed="1" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <Output>
      <StdOut>[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.3.1+6b60a9e56a (64-bit .NET 8.0.7)&#xD;
[xUnit.net 00:00:00.12]   Discovering: MyCompany.ConsumerApi.IntegrationTest&#xD;
[xUnit.net 00:00:00.21]   Discovered:  MyCompany.ConsumerApi.IntegrationTest&#xD;
[xUnit.net 00:00:00.22]   Starting:    MyCompany.ConsumerApi.IntegrationTest&#xD;
[xUnit.net 00:00:02.21]   Finished:    MyCompany.ConsumerApi.IntegrationTest&#xD;
</StdOut>
    </Output>
  </ResultSummary>
</TestRun>

The stdout is as expected. The number of test results is as expected. Everything is nice and fine.

I have many trx files (this is the case for good reasons that I do not want to discuss here). But Azure Pipelines does not see my test results when publishing them.

- task: PublishTestResults@2
  inputs:
    testResultsFiles: '**/*.trx'
    searchFolder: '$(System.DefaultWorkingDirectory)'
    testRunTitle: 'Test Results'
    publishRunAttachments: true
    failTaskOnFailedTests: true  # This ensures the task fails if any test fails

I got an output like this

2024-07-22T10:20:08.6511447Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_37.trx'.
2024-07-22T10:20:08.6512662Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_37[1].trx'.
2024-07-22T10:20:08.6526375Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_40.trx'.
2024-07-22T10:20:08.6527626Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_41.trx'.
2024-07-22T10:20:08.6632274Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_44.trx'.
2024-07-22T10:20:08.6633882Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_46.trx'.
2024-07-22T10:20:08.6635084Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_48.trx'.
2024-07-22T10:20:08.6637200Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_50.trx'.
2024-07-22T10:20:08.6638387Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_52.trx'.
2024-07-22T10:20:08.6639614Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_54.trx'.
2024-07-22T10:20:08.6640816Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_56.trx'.
2024-07-22T10:20:08.6642046Z No Result Found to Publish 'D:\a\1\s\TestResults\VssAdministrator_fv-az520-777_2024-07-22_10_19_58.trx'

It reality, it is longer but all the time I got "No Result Found to Publish". This is extremely frustrating.

Let me explain what goes well:

My tests succeed
I am happy with my trx
The publish step recognizes all trx files to be published.
The trx files really have the right content and have test results in them.

What does not go well.

The publish PublishTestResults@2 complains that it cannot find the test results.

This is not true. It can be seen easily that the rest results can be found.

I thus need to find another solution to make sure my test results are properly published in Azure pipelines. What is that other solution?

In case it helps answering the question: Here is an example of a (generated and executed) "dotnet test" command.

dotnet test -c Release --no-restore --no-build  "SOME SOLUTIONFILE" --filter FullyQualifiedName='SOMETESTNAME' -p:Platform=AnyCPU --results-directory "./TestResults" --logger "trx" -- RunConfiguration.TreatNoTestsAsError=True

Solution

You're not setting the testResultsFormat property - its default value is Junit, that's probably the reason why tests are not published.

Allowed values are the following:

JUnit
NUnit
VSTest
XUnit
CTest

Try something like:

- task: PublishTestResults@2
  inputs:
    testResultsFiles: '**/*.trx'
    testResultsFormat: 'VSTest' # <------------------------ Set format here
    searchFolder: '$(System.DefaultWorkingDirectory)'
    testRunTitle: 'Test Results'
    publishRunAttachments: true
    failTaskOnFailedTests: true  # This ensures the task fails if any test fails

See PublishTestResults@2 for more details.