Search code examples
c#.net-corecsprojxunit.net

Does Xunit use a source generator for test data?

My tests are something like this:

[Theory]
[MemberData(nameof(TestData))]
public async Task Test(DateTime? startDate)
{
    // Act
    var currentDate = RunCalculation();

    // Assert
    currentDate.Should().BeCloseTo(startDate, TimeSpan.FromSeconds(5));
}

public static IEnumerable<object[]> TestData()
{
    var now = DateTime.UtcNow;

    return new[]
    {
        new object[]
        {
            now,
        },
        new object[]
        {
            now,
        },
    }
}

currentDate doesn't depend on startDate, but after the test has run, they should be similar. The problem is not with currentDate, but with startDate.

The test appears to be called with the UtcNow of when the code was built, not when the code is executed. If I build the code, then run the test 5 minutes later, the test fails. If I build and run the test immediately, it succeeds.

Does Xunit somehow autogenerate the test data at compile time, with fixed values? Is there something else at play here?

The csproj files also have <AccelerateBuildsInVisualStudio>true</AccelerateBuildsInVisualStudio>, would that effect something?

Versions:

  • NET 6
  • VS2022 17.9.2
  • xunit 2.6.5
Solution

  • No, there's no source generator being used.

    In short without going technical, the DateTime.Now is being 'fixed' -- it's encoded into the test case -- at discovery time, subsequent runs will keep using that one, until a new build occurs.

    xUnit has a "Theory Data Stability in Visual Studio" topic about that here, especially focusing on DateTime.Now.

    You'll notice that the value shown in Visual Studio Test Explorer doesn't change between runs, but only when you rebuild.

    enter image description here

    To overcome this, you can include DisableDiscoveryEnumeration = true in the MemberDataAttribute as shown below. See the source code on GitHub.

    [MemberData(nameof(TestData), DisableDiscoveryEnumeration = true)]

    That same document considers this a bug in Visual Studio and makes the alternative suggestion to use an other test runner:

    The first choice is to use a different runner. To the best of my knowledge, none of the other runners other than Visual Studio Test Explorer exhibit this behavior.