Search code examples
profilingcode-coverage

difference between code coverage and profiling


What is difference between code code coverage and profiling.

Which is the best open source tool for code coverage.


Solution

  • Coverage is important to see which parts of the code have not been run. In my experience it has to be accumulated over multiple use cases, because any single run of the software will only use some of the code.

    Profiling means different things at different times. Sometimes it means measuring performance. Sometimes it means diagnosing memory leaks. Sometimes it means getting visibility into multi-threading or other low-level activities.

    When the goal is to improve software performance, by finding so-called "bottlenecks" and fixing them, don't just settle for any profiler, not even necessarily a highly-recommended or venerable one. It is essential to use the kind that gets the right kind of information and presents it to you the right way, because there is a lot of confusion about this. More on that subject.

    Added: For a coverage tool, I've always done it myself. In nearly every routine and basic block, I insert a call like this: Utils.CovTest("file name, routine name, comment that tells what's being done here"). The routine records the fact that it was called, and when the program finishes, all those comments are appended to a text file. Then there's a post-processing step where that file is "subtracted" from a complete list (gotten by a grep-like program). The result is a list of what hasn't been tested, requiring additional test cases.

    When not doing coverage testing, Utils.CovTest does nothing. I leave it out of the innermost loops anyway, so it doesn't affect performance much. In C and C++, I do it with a macro that, during normal use, expands to nothing.