This article continues the series on building a continuous deployment environment using Python and Django.

• Starting Your First Django Project
• Testing and Django
• Mock and Coverage
• Using Fabric for Painless Scripting
• Using Celery to handle Asynchronous Processes
• Deployment/Monitoring Strategies

So far we have covered the basics of setting up a Django project and testing it. Today we will discuss how to ensure your tests fully cover the application using the Python mock and coverage tools.

Getting ready

Coverage is a tool that indicates the lines of code executed when a command runs, such as manage.py test. To install coverage (version 3.4 was the latest when writing this article), enter your project's context, and run :

pip install coverage

Mock is a tool that allows developers to specify fake responses for parts of their code. This is especially useful when testing code that requests remote services, such as facebook or twitter. To install mock (version 0.7.2 was the latest when writing this article), enter your project's context, and run:

pip install mock

How to do it…

Coverage

Using coverage is really easy, simply run the test management command through it:

coverage run manage.py test
coverage report

You may also output the report in HTML or XML formats:

coverage html
coverage xml

Using coverage this way works, as long as you ensure each file in the project is imported by the tests, because coverage will only report on files that are loaded during the run command. Unfortunately, this technique is less than ideal when testing a single app, as it will test a lot of files that aren't related to the app.

Fortunately, there is a Django coverage tool (version 1.2 was the latest when writing this article):

pip install django-coverage

You can then configure coverage to run in settings.py, making coverage report more useful results, especially when testing a single app. Additionally, there are configuration options that provide greater control over how coverage evaluates the project. I don't change any of the default options, see django_coverage/settings.py for all available settings. To enable coverage, add the following to your settings.py:

# not changed, just shown as an example
COVERAGE_MODULE_EXCLUDES = [
        'tests$', 'settings$', '^urls$', 'locale$',
        '__init__', 'django',
        'migrations',
]

import coverage
coverage.use_cache(False)
for e in COVERAGE_CODE_EXCLUDES:
    coverage.exclude(e)
coverage.start()

TEST_RUNNER = 'django_coverage.coverage_runner.CoverageRunner'

First modify any coverage properties that necessary for your project, although the defaults are comprehensive. Then start coverage right away; normally coverage is started in the test runner, however I start it here, because function definitions will be reported as uncovered lines of code if you start the process later. Lastly, define the CoverageRunner as the project's TEST_RUNNER.

Now code coverage will be evaluated each time you run a test, and running tests for a specific app will only report on files related to that app. To turn coverage off, simply comment out the code you added to settings.py. Unfortunately, because coverage is turned on in settings.py, there isn't a good way to turn it on from the command line without modifying manage.py.

When using coverage try making your apps at least 90% covered. This will ensure that your app behaves as expected. Getting to 100% is a great goal, but sometimes it is exceptionally tedious, especially when you have try/except statements around code that should never fail, so the except can't be triggered without using a Mock tool.

Using Mock

Mock can be used when defining a test function or called explicitly to override a function or class that is being imported by some part of the project. Say you want to change how a function behaves in one of your test cases, you would use mock to patch it:

import mock
from django.test import TestCase

class TestUsingMock(TestCase):
    @mock.patch('django.core.urlresolvers.reverse')
    def test_reverse(self, mock_reverse):
    	from django.core.urlresolvers import reverse
    	mock_value = 'www.mattsnider.com'
    	mock_reverse.return_value = mock_value
    	self.assertEquals(mock_value, reverse('1234'))
    	self.assertEquals(mock_value, reverse('abcd'))

In this example we mock the django.core.urlresolvers.reverse function to return a mock url, instead of actually evaluating against the provided argument. To mock the function, use the patch decorator on your test function. The mock object will be passed in as the second argument. You can patch as many functions as needed, and they will be passed into the function as arguments in the order they were patched. When patching define the return_value on the mock instance, which will be the value returned any time the mocked function is called (throughout the project, not just inside the test function) while in the execution context of the patched function.

You may also patch the value of any object that you have access to, such as django.conf.settings:

import mock
from django.conf import settings
from django.test import TestCase

class TestUsingMock(TestCase):
    @mock.patch.object(settings, 'TEST_RUNNER', 'my_test_runner')
    def test_reverse(self, mock_reverse):
    	from django.conf import settings
    	self.assertEquals('my_test_runner', settings.TEST_RUNNER

With these two patching techniques you can now cover 100% of your application, however there is a lot more you can do with mock, such as setting argument expectations and patching the with statement. For more details, see:

Mock - Mocking and Testing Library

There's more…

Occasionally, even with 100% coverage of your code, you will have errors. When this happens, first write a test that fails, while reproducing the error. Then when you fix the code, the test will also pass. In this way you can improve your tests as your codebase grows and prevent regressions.

4 Comments

Tags: test, django, Python