Currently have a project configured to run coverage via Django's manage command like so:
./manage.py test --with-coverage --cover-package=notify --cover-branches --cover-inclusive --cover-erase
This results in a report like the following:
Name Stmts Miss Branch BrMiss Cover Missing
--------------------------------------------------------------------------
notify.decorators 4 1 0 0 75% 4
notify.handlers 6 1 2 0 88% 11
notify.notification_types 46 39 2 0 19% 8-55, 59, 62, 66
notify.notifications 51 51 0 0 0% 11-141
--------------------------------------------------------------------------
TOTAL 107 92 4 0 17%
There's a problem with this report however. It's wrong. Coverage is marking lines missing, despite the fact that they are indeed being covered by tests. For example, if I run the tests via nosetests instead of django's manage command I get the following correct report:
Name Stmts Miss Branch BrMiss Cover Missing
-----------------------------------------------------------------------------
notify.decorators 4 0 0 0 100%
notify.handlers 6 0 2 0 100%
notify.notification_types 46 0 2 0 100%
notify.notifications 51 25 0 0 51% 13, 18, 23, 28, 33, 38, 43, 48, 53, 65, 70, 75, 80, 85, 90, 95, 100, 105, 110, 116, 121, 126, 131, 136, 141
-----------------------------------------------------------------------------
TOTAL 107 25 4 0 77%
Google led me to the coverage website's FAQ, http://nedbatchelder.com/code/coverage/faq.html
Q: Why do the bodies of functions (or classes) show as executed, but the def lines do not?
This happens because coverage is started after the functions are defined. The definition lines are executed without coverage measurement, then coverage is started, then the function is called. This means the body is measured, but the definition of the function itself is not.
To fix this, start coverage earlier. If you use the command line to run your program with coverage, then your entire program will be monitored. If you are using the API, you need to call coverage.start() before importing the modules that define your functions.
The question is, can I run the coverage reports properly via Django's manage command? Or do I have to bypass manage to avoid the situation where coverage is started after the "missing" lines are executed?
At the moment it's not possible to accurately run coverage alongside with django-nose (because of the way Django 1.7 loads models). So to get the coverage stats, you need to use coverage.py directly from command line, e.g:
$ coverage run --branch --source=app1,app2 ./manage.py test
$ coverage report
$ coverage html -d coverage-report
You can put coverage.py settings into .coveragerc file in the project root (the same dir as manage.py).
This issue is reported on django-nose GitHub page: https://github.com/django-nose/django-nose/issues/180 so maintainers know about the problem, you can let them know that you're also experiencing this issue.
UPDATE
eliangcs pointed out (django-nose issues on GiHub), that woraround is to modify your manage.py:
import os
import sys
if __name__ == "__main__":
# ...
from django.core.management import execute_from_command_line
is_testing = 'test' in sys.argv
if is_testing:
import coverage
cov = coverage.coverage(source=['package1', 'package2'], omit=['*/tests/*'])
cov.erase()
cov.start()
execute_from_command_line(sys.argv)
if is_testing:
cov.stop()
cov.save()
cov.report()
It works, but it's rather "hacky" approach.
UPDATE 2
I recommend for everybody that uses nose to have a look at py.test (http://pytest.org/), which is really good Python testing tool, it integrates well with Django, has a lot of plugins and many more. I was using django-nose, but tried py.test and never looked back.