Search code examples
c#.netcontinuous-integrationteamcityapproval-tests

How to use ApprovalTests on Teamcity?

I am using Approval Tests. On my dev machine I am happy with DiffReporter that starts TortoiseDiff when my test results differ from approved:

    [UseReporter(typeof (DiffReporter))]
    public class MyApprovalTests
    { ... }

However when the same tests are running on Teamcity and results are different tests fail with the following error:

System.Exception : Unable to launch: tortoisemerge.exe with arguments ...
Error Message: The system cannot find the file specified
---- System.ComponentModel.Win32Exception : The system cannot find the file 
                                                                 specified

Obviously it cannot find tortoisemerge.exe and that is fine because it is not installed on build agent. But what if it gets installed? Then for each fail another instance of tortoisemerge.exe will start and nobody will close it. Eventually tons of tortoisemerge.exe instances will kill our servers :)

So the question is -- how tests should be decorated to run Tortoise Diff on local machine and just report errors on build server? I am aware of #IF DEBUG [UseReporter(typeof (DiffReporter))] but would prefer another solution if possible.

Solution

  • There are a couple of solutions to the question of Reporters and CI. I will list them all, then point to a better solution, which is not quite enabled yet.

    1. Use the AppConfigReporter. This allows you to set the reporter in your AppConfig, and you can use the QuietReporter for CI. There is a video here, along with many other reporters. The AppConfigReporter appears at 6:00. This has the advantage of separate configs, and you can decorate at the assembly level, but has the disadvantage of if you override at the class/method level, you still have the issue.

    2. Create your own (2) reporters. It is worth noting that if you use a reporter, it will get called, regardless as to if it is working in the environment. IEnvironmentAwareReporter allows for composite reporters, but will not prevent a direct call to the reporter. Most likely you will need 2 reporters, one which does nothing (like a quiet reporter) but only works on your CI server, or when called by TeamCity. Will call it the TeamCity Reporter. And One, which is a multiReporter which Calls teamCity if it is working, otherwise defers to .

    3. Use a FrontLoadedReporter (not quite ready). This is how ApprovalTests currently uses NCrunch. It does the above method in front of whatever is loaded in your UseReporter attribute. I have been meaning to add an assembly level attribute for configuring this, but haven't yet (sorry) I will try to add this very soon.

    Hope this helps. Llewellyn