C# xUnit Test listeners

Question

I'm building a selenium test framework based on .Net Core and the team decided to go with xUnit. All's well and good everything has been going ok but for a while now, we've been trying to replicate the functionality of Java TestNG listeners without much luck.

I've been digging around the xunit git repo and found a few instances where some interfaces such ITestListener have been used. After digging deeper, I found that these listeners are from a package called TestDriven.Framework and I wanted to know exactly how would I use a test listener created using those interfaces?

So far this is my simple test listener that should write something when the test fails:

public class Listener
{
    readonly int totalTests;

    public Listener(ITestListener listener, int totalTests)
    {
        this.totalTests = totalTests;
        TestListener = listener;
        TestRunState = TestRunState.NoTests;
    }

    public ITestListener TestListener { get; private set; }
    public TestRunState TestRunState { get; set; }

    public void onTestFail(ITestFailed args)
    {
        Console.WriteLine(args.Messages);
    }

}

Now, I know you can do this inside a tear down hook but remember, this is just a simple example and what I have in mind is something more complex. So to be precise, where/how exactly would I register the test to use this listener? In Java TestNg I would have @Listeners but in C# I'm not too sure.

Edit 1 so the example worked and managed to add it to my own project structure but when I try to use this

    class TestPassed : TestResultMessage, ITestPassed
{
    /// <summary>
    /// Initializes a new instance of the <see cref="TestPassed"/> class.
    /// </summary>
    public TestPassed(ITest test, decimal executionTime, string output)
        : base(test, executionTime, output) {
        Console.WriteLine("Execution time was an awesome " + executionTime);
    }
}

I'm having trouble registering this one, or if i'm even registering it right. As far as the examples go, I have found the actual message sinks but also found the actual test status data which i'm not exactly sure how to use.

Answer 1

I haven't worked with TestNG, but I did some quick reading and I think I see what you're after.

To demonstrate, I've created a very basic proof-of-concept implementation of the xUnit IMessageSink interface.

public class MyMessageSink : IMessageSink
{
    public bool OnMessage(IMessageSinkMessage message)
    {
        // Do what you want to in response to events here.
        // 
        // Each event has a corresponding implementation of IMessageSinkMessage.
        // See examples here: https://github.com/xunit/abstractions.xunit/tree/master/src/xunit.abstractions/Messages
        if (message is ITestPassed)
        {
            // Beware that this message won't actually appear in the Visual Studio Test Output console.
            // It's just here as an example. You can set a breakpoint to see that the line is hit.
            Console.WriteLine("Execution time was an awesome " + ((ITestPassed)message).ExecutionTime);
        }

        // Return `false` if you want to interrupt test execution.
        return true;
    }
}

The sink is then registered via an IRunnerReporter:

public class MyRunnerReporter : IRunnerReporter
{
    public string Description => "My custom runner reporter";

    // Hard-coding `true` means this reporter will always be enabled.
    //
    // You can also implement logic to conditional enable/disable the reporter.
    // Most reporters based this decision on an environment variable.
    // Eg: https://github.com/xunit/xunit/blob/cbf28f6d911747fc2bcd64b6f57663aecac91a4c/src/xunit.runner.reporters/TeamCityReporter.cs#L11
    public bool IsEnvironmentallyEnabled => true;
        
    public string RunnerSwitch => "mycustomrunnerreporter";

    public IMessageSink CreateMessageHandler(IRunnerLogger logger)
    {
        return new MyMessageSink();
    }
}

To use my example code, just copy the classes into your test project (you'll also need to add a reference to the xunit.runner.utility NuGet package). The xUnit framework will automagically discover the IRunnerReporter--no need to explicitly register anything.

If this seems like it's headed in the right direction, you can find a lot more info in the xUnit source code. All of the interfaces involved are well-documented. There are a few existing implementations in the xunit.runner.reporters namespace. AssemblyRunner.cs also demonstrates one possible method for dispatching the different event types to individual handlers.

Edit 1

I've updated the implementation of MyMessageSink (above) to demonstrate how you might listen for an ITestPassed message. I also updated the link embedded in that code snippet--the previous link was to implementations, but we should really use these abstractions.

The if (message is IMessageType) pattern is pretty crude, and won't scale well if you want to listen for many different message types. Since I don't know your needs, I just went with the simplest thing that could possibly work--hopefully it's enough that you can improve/extend it to fit your needs.