Search code examples
unit-testinglanguage-agnosticdefects

Is there a type of logic/flow errors that cannot be discovered by unit tests?


Assuming I have contracts defined and clear requirements (covering input ranges, boundary values etc.), and unit tests validating all those conditions, can there still be integration errors when I put those units together? I do not consider external services now. I have seen the following as an example of integration error but I believe this is just a missing test at unit level:

class Engine
{
   int RPM;

   void SetRPMtoZero()
   {
      RPM=0;
   }
}

class Display
{
  CalculateAverage(Engine e)
  {
    if (e.IsRunning)
    {
      int X=smth/e.RPM;  //could be division by 0
    }
  }
}

class IntegratingClass
{
  Engine e
  Display d..

  ...

  e.SetRPMtoZero();
  d.CalculateAverage(e);

  //this sequence would lead to the division by zero

}

I do not think this shows an integration error - the CalculateAverage simply lacks the check for RPM!=0.

Is there actually a type of logical errors (or control flow) that cannot by discovered by unit testing?


Solution

  • I agree with you that the example rather is about a missing unit test for the class Display.

    However, maybe the author of that example wanted to point out that the coupling between units can be stronger than expected or than described in a contract. Furthermore this example makes already clear that on integration level things can be different than expected on unit test level (maybe it was not intended that the SetRPMtoZero method is called just before the CalculateAverage method). Usually you don't see that on unit test level, unless you have the best specification in the world.

    So, what else you may encounter when integrating your units (at the same time this is also what unit-tests can't tell you):

    • Even on the first stage of integration test, the integration itself (f.e. linking the units together, if we talk about a language like c++), can fail, if the interface changed in one unit, but not adapted accordingly in the other(s).
    • If the units use common resources (like a file, memory), then reservation/release of a resource may cause blocked states or data corruption
    • If you cover the source code of a unit by unit test with 100% than this does not mean that all of the code is really necessary. On integration level you may realize that there is lot of code, which can't even be reached, when putting the units together. This can help you to find out, which parts of your code are probably "dead code".
    • Performance restrictions or resource limits (like limited memory)
    • Misinterpretation of the contract or errors in its implementation