Search code examples
c#unit-testingroslyncode-analysis

Roslyn code-fix test calls `VerifyDiagnostics` also for fixed-code-sample, which makes test could never be successfull



#UPD: It was totally my mistake. Some details I've posted in my answer


Why does Roslyn code-fix test calls VerifyDiagnostics not only for test-source-code-sample, but then also for fixed-code-sample?

Unit tests of the Visual Studio template project "Analyzer with Code Fix" rely on such method:

public static async Task VerifyCodeFixAsync(
    /*1*/ string source, 
    /*2*/ DiagnosticResult[] expected, 
    /*3*/ string fixedSource)
{
    var test = new Test
    {
        TestCode = source,
        FixedCode = fixedSource,
    };

    test.ExpectedDiagnostics.AddRange(expected);
    await test.RunAsync(CancellationToken.None);
}

I understand arguments this way:

/*1*/ - test code sample, with deliberate problems to test analyzer can find them

/*2*/ - expecteds - special data about these deliberate problems (incl. location of each problem)

/*3*/ - fixed code sample - it's what test code sample shoud became looks after applying auto-code fixing to it at places, analyzer have found

So, I expect test.RunAsync (called at last line) should work uder the hood in scenario similar to:

  • 1st compare problems found by analyzer with expecteds - they should be equal
  • 2nd apply auto-code fixing to the test-code-sample
  • and finally compare code, produced by prev. step, with provided fixedSource (/*3*/)

And that's all I expect.

BUT! If you look at source code of class Microsoft.CodeAnalysis.Testing.CodeFixTest<> you could find such implementation of RunAsync method:

public override async Task RunAsync(CancellationToken cancellationToken = default)
{
    ...
    await VerifyDiagnosticsAsync(new EvaluatedProjectState(testState, ...);  // <--- Line A

    if (CodeFixExpected())
    {
        await VerifyDiagnosticsAsync(new EvaluatedProjectState(fixedState, ...);  // <--- Line B
        ...
    }
}

It results in case of my analyzer to:

  • If I pass not-empty expected, test fails at Line B with message:

Assert.AreEqual failed. Expected:<1>. Actual:<0>. Context: Diagnostics of fixed state

'cos it looks like it somewhy compares problems found in fixed code sample to expected (which, I suppose is a mistake, as fixed code should not contain problems (by definition))


#UPD: Roslyn code is correct. It was my mistake - see at my posted answer.


  • If I pass empty expected, Line B should be satisfied, but test fails (quite expected this time) at Line A with message:

Assert.AreEqual failed. Expected:<0>. Actual:<1>. Context: Diagnostics of test state

So my question is: Why does Roslyn code-fix test compares expected to both (1st to test code sample, as completely expected as for me), but then to fixed code sample, which makes test could never be successfull?

Or maybe I'm missing something...

I would be grateful for any thoughts


Solution

  • I figured it out. It was my fault. By mistake I used CSharpCodeFixVerifier parametrized with another AnalyzerCodeFixProvider (not the one I should use there).

    So, that 2nd call of VerifyDiagnostics (with fixed-code-sample as state argument) is correct. And, I guess, it checks that fixed-code-sample has no diagnosed problems.