Search code examples
c#.netfileunit-testingintegration-testing

How to do integration testing in .NET with real files?

I have some classes that implements some logic related to file system and files. For example, I am performing following tasks as part of this logic:

  • checking if certain folder has certain structure (eg. it contains subfolders with specific names etc...)
  • loading some files from those folders and checking their structure (eg. these are some configuration files, located at certain place within certain folder)
  • load additional files for testing/validation from the configuration file (eg. this config file contains information about other files in the same folder, that should have other internal structure etc...)

Now all this logic has some workflow and exceptions are thrown, if something is not right (eg. configuration file is not found at the specific folder location). In addition, there is Managed Extensibility Framework (MEF) involved in this logic, because some of these files I am checking are managed DLLs that I am manually loading to MEF aggregates etc...

Now I'd like to test all this in some way. I was thinking of creating several physical test folders on HDD, that cover various test cases and then run my code against them. I could create for example:

  • folder with correct structure and all files being valid
  • folder with correct structure but with invalid configuration file
  • folder with correct structure but missing configuration file etc...

Would this be the right approach? I am not sure though how exactly to run my code in this scenario... I certainly don't want to run the whole application and point it to check these mocked folders. Should I use some unit testing framework to write kind of "unit tests", that executes my code against these file system objects?

In general, is all this a correct approach for this kind of testing scenarios? Are there other better approaches?

Solution

  • First of all, I think, it is better to write unit tests to test your logic without touching any external resources. Here you have two options:

    1. you need to use abstraction layer to isolate your logic from external dependencies such as the file system. You can easily stub or mock (by hand or with help of constrained isolation framework such as NSubstitute, FakeItEasy or Moq) this abstractions in unit tests. I prefer this option, because in this case tests push you to a better design.
    2. if you have to deal with legacy code (only in this case), you can use one of the unconstrained isolation frameworks (such as TypeMock Isolator, JustMock or Microsoft Fakes) that can stub/mock pretty much everything (for instance, sealed and static classes, non-virtual methods). But they costs money. The only "free" option is Microsoft Fakes unless you are the happy owner of Visual Studio 2012/2013 Premium/Ultimate.

    In unit tests you don't need to test the logic of external libraries such as MEF.

    Secondly, if you want to write integration tests, then you need to write "happy path" test (when everything is OK) and some tests that testing your logic in boundary cases (file or directory not found). Unlike @Sergey Berezovskiy, I recommend creating separate folders for each test case. The main advantages is:

    1. you can give your folder meaningful names that more clearly express your intentions;
    2. you don't need to write complex (i.e. fragile) setup/teardown logic.
    3. even if you decide later to use another folder structure, then you can change it more easily, because you will already have working code and tests (refactoring under test harness is much easier).

    For both, unit and integration tests, you can use ordinary unit testing frameworks (like NUnit or xUnit.NET). With this frameworks is pretty easy to launch your tests in Continuous integration scenarios on your Build server.

    If you decide to write both kinds of tests, then you need to separate unit tests from integration tests (you can create separate projects for every kind of tests). Reasons for it:

    1. unit tests is a safety net for developers. They must provide quick feedback about expected behavior of system units after last code changes (bug fixes, new features). If they are run frequently, then developer can quickly and easily identify piece of code, that broke the system. Nobody wants to run slow unit tests.
    2. integration tests are generally slower than unit tests. But they have different purpose. They check that units works as expected with real dependencies.