Search code examples
c#xamarinmonocode-coveragemvvmcross

How to obtain code coverage with Xamarin MvvmCross?

tl;dr

Hi! I'm trying to find out what is the code coverage for our Xamarin application (Xamarin.Android) developed using MvvmCross with Visual Studio Community 2017 in Windows 10.

Description

We tried to do so twice already:

  • The first time we created a MSTest project, in which we were able to run in Test Explorer of Visual Studio and also use SonnarQube to export the outputs. Although we could inspect our Code Duplication, Bugs, Vulnerabilities and Code Smells, SonarQube simply doesn't report the Code Coverage.

We realized it was because Xamarin projects use .NET Standard instead of .NET Framework, and SonarQube only supports .NET Framework. Then we found this link with a solution for Mac, which I quote:

It looks like the best solution we have come across is to compile the tests into an executable with NUnitLite runner and then use the log profiler, mprof-report and ReportGenerator to generate the final html files :

e.g.

mono --debug --profile=log:coverage,covfilter=+MyNamespace,covfilter=-NotMyNamespace,output=coverage.mlpd MyTests.exe --noresult

mprof-report --reports=coverage --coverage-out=coverage.xml coverage.mlpd

mono ./packages/ReportGenerator.2.5.6/tools/ReportGenerator.exe -reports:coverage.xml -targetdir:report

firefox ./report/index.htm

  • So we converted our test project to use NUnit. We were able to attach NUnit to Visual Studio with the NUnit.Adapter, we were able to make an executable out from it and installed Mono for Windows (x64), from the binary installer (*.msi). As we run the first command, it returns the results from the tests but also gives an error:

The 'log' profiler wasn't found in the main executable nor could it be loaded from 'mono-profiler-log'.

Therefore, there is no coverage output. As a matter of fact, it seems like there is no mprof-report installed also, even in the Mono Command Prompt.

So, is there ANY way at all to find out what is the code coverage for our Xamarin MvvmCross project, considering we are not in any way inclined to buy enterprise editions of software? If so, how can it be done?

Solution

  • Finally got it! It's definitely not CI/CD friendly at all (yet), but it can be used to at least know the current state of unit tests in an MvvmCross Xamarin application.

    Also, it does not cover the Droid project. But as we're using MVVM, the Droid project should only contain Views, resources and Bindings anyway (ideally). So, in my opinion, it's not as much important to cover it as the other projects (AppCore, Services, etc.)...

    Anyway, here's what I did:

    • Install NUnit, NUnitLite, NUnit3TestAdapter, OpenCover and ReportGenerator (all of them via NuGet, directly from Visual Studio)
    • Convert MSTest to NUnit
    • Build tests into an executable console application with NUnitLite
    • *Change the debugType in the buildOptions for each and every project in the application solution (except the Droid project and any test project you have) from "Portable" to "Full": Go to Project Properties -> Build. Then, on Output Section click Advanced and select Debugging Information -> Change to Full. (Thanks, m4ss1m0g!)
    • Now, run the OpenCover and the ReportGenerator commands (or the modified .bat script below) to have your coverage report generated into an human readable .htm file. Open the .htm file with your browser, and - voilà - you have your coverage reported!

    Here is the simple script I used, which I placed in my solution's root folder (Attention to the version of the OpenCover and ReportGenerator packages, and all the paths used! You can also place filters with OpenCover to exclude or include any Namespaces, Classes and Methods you want).

    packages\OpenCover.4.6.519\tools\OpenCover.Console.exe -skipautoprops -output:<path-where-you-want-your-xml-results-to-be>\coverage.xml -target:<path-to-your-test-executable-generated-by-NUnitLite> -register:user -filter:"+[*]<your-solution-name>.* -[nunit*]* -[*<your-test-project-name>]*"

    packages\ReportGenerator.4.0.4\tools\net47\ReportGenerator.exe -reports:<path-where-your-xml-results-output-are--same-as-output-path-above>\coverage.xml -targetdir:coverage

    Obs:

    1. After following the steps described above and confirming it works, you can install the AxoCover VisualStudio extension, which offers a practical way to see your current code coverage, instead of having to run the .bat script above and navigating to the folder where the output coverage.htm is generated.
    2. The third step in the process described above (marked with an *) is key to make sure your test executable consider your other projects, besides your test project.