Search code examples
c#.netmstest

MS Test giving inconsistent results for simple tests

The following tests are detected differently by the Visual Studio Test Explorer and have different output formats despite their similarities.

[TestClass]
public class UnitTest1
{
    [TestMethod("Test Method")]
    [DataRow("String#1", 1, DisplayName = "String#1-1 Display Name")]
    [DataRow("String#2", 2, DisplayName = "String#2-2 Display Name")]
    public void Test(string s, int i)
    {
        Console.WriteLine($"{s}-{i}");
    }

    [TestMethod("Test Method with Dummy object")]
    [DataRow("String#1", 1, typeof(object), DisplayName = "String#1-1 Display Name")]
    [DataRow("String#2", 2, typeof(object), DisplayName = "String#2-2 Display Name")]
    public void TestWithDummy(string s, int i, object dummy)
    {
        Console.WriteLine($"{s}-{i}-{dummy}");
    }
}

Test Explorer Display

The output for the method Test gives a separate Test Detail Summary for each DataRow.

The output for the method TestWithDummy uses the display name and displays a Test Detail Summary that shows all the results.

TestWithDummy - Test Detail Summary

Why would adding a dummy object parameter cause the second test to concatenate its results together? This looks like a far more useful output format.

I'm using VS2022 17.6.5, .NET 7, MS Test SDK 17.7.0, MS Test Adapter and TestFramework 3.1.1.

Solution

  • It looks like you found a bug. Looking around the MSTest repo I found one open issue that is very similar to yours: TestMethod displayName is being ignored if DataRowAttribute is specified.

    I'd suggest that you ask in that ticket if they consider what you found as the same bug, and if not, open a new ticket.