Example of a Regression Test & Unit Test - Windows Forms LOB application

Question

We have a LOB Winforms application that follows MVP pattern as much as possible.

At each new release, we want to test against any regression.

Current approach is that every time a tester a bug/crash, we reproduce the bug, then insert a test using NUnit. Usually this test tries to reproduce the chain of actions followed by the user, so we end up with tests like : MenuX_OperationY_buttonDoZ_Click_ERROR_DESCRIPTION()

The test usually prepares the context, load data, and execute the action clicked by the user to reproduce the bug. Then the bug is fixed and the test is rerun. And so on ..., until we have a large base of existing (regression ?) tests. Example of such a test is :

[Test]
public void MenuX_OperationY_buttonDoZ_Click_ERROR_DESCRIPTION()
{
    // The presenter prepares Context & Load Data associated with Menu X
    PrsMenuX.PrepareContext();
    PrsMenuX.LoadData();

    // Do Operation Y
    PrsMenuX.OperationY();

    // Click on Button Do Z
    try
    {
        PrsMenuX.ButtonDoZ_Click();
    }
    catch (Exception ex)
    {
        Assert.Fail(ex.Message);
    }
}

So my question, is it the right approach ? is NUnit the right tool for this, and how to improve this ? Should the tests reproduces the user's actions or should we unit-test only low level functions that composes each user's action ?

Answer 1

It depends on what is failing. If it is a bug in a algorithm in your code which is failing, then it probably should be tested by an unit test. If it is more of an unexpected usage bug, then it should be a more high level acceptance/behaviour test. And just because you add an unit test doesn't mean that you should not add a more high level test.

Also it seems a bit late to generally add tests only after bugs have been found. It would be better if the tests were written when a new feature was added, or existing feature was edited. It is generally easier to write good tests the same time the code is written, as opposed to writing a test for unknown or forgotten code at a later time. Practicing this will keep more bugs from not being introduced in the first place.