Setting up pymysql to work with django, pytest and pytest-django

Question

I have a Django website set up along with tests based on the pytest and pytest-django packages. I wish to get the PyMySQL package to work with my setup for testing and production.

I understand that in order to get PyMySQL to expose itself as a mysqldb module so django (and other modules) can work with it I should place the following snippet at the beginning of my code.

import pymysql
pymysql.install_as_MySQLdb()

I made several attempts to call above function at different locations throughout my code but none seem to be detected by pytest early enough.

This includes placing the snippet inside conftest.py:pytest_configure, in my django project's manage.py file, etc. Neither worked.

The reason I'm switching from mysqlclient to PyMySQL is that mysqlclient doesn't seem to be functioning with travis.org once I include the following in .travis.yml:

addons:
  mariadb: '10.0'

In order to use MariaDB instead of vanilla MySQL.

Getting travis to work with the python mysqlclient package will also solve my problem, however I could not get that done either.

For reference, this is a travis job for the PR, in case I missed something important out.

Answer 1

Well, at first I thought that's impossible to do because pytest-django imports a lot of django stuff on module level already, and you can't beat that by plugging yourself via hooks - they are all executed after the plugins are loaded via entry points, so django will be already imported by then.

However, looking at pytest --help, notice a rarely used option that turns out to be helpful in this case:

$ pytest --help
...
test session debugging and configuration:
  -p name               early-load given plugin (multi-allowed). To avoid
                        loading of plugins, use the `no:` prefix, e.g.
                        `no:doctest`.

Run pymysql configuration via plugin early loading

So custom plugins passed via -p arg will be imported earlier than the ones loaded via entrypoints. The solution is now easy: write a small module that imports and configures pymysql and pass it via -p arg:

# rematch/pymysql_hack.py
import pymysql
pymysql.install_as_MySQLdb()

Tests invocation:

$ PYTHONPATH=. pytest -p pymysql_hack -rapP server tests/server

I forked the rematch project to test this out, the jobs are looking good (the first job fails because I was too lazy to add pymysql in all the requirements files).

Give up pytest, use manage.py test

Alternatively (this was my first solution proposal), you can run tests via manage.py, so you could add the pymysql configuration lines on top of manage.py, implement a custom test runner as described in pytest-django's FAQ:

pytest-django is designed to work with the pytest command, but if you really need integration with manage.py test, you can create a simple test runner like this:

class PytestTestRunner(object):
    """Runs pytest to discover and run tests."""

    def __init__(self, verbosity=1, failfast=False, keepdb=False, **kwargs):
        self.verbosity = verbosity
        self.failfast = failfast
        self.keepdb = keepdb

    def run_tests(self, test_labels):
        """Run pytest and return the exitcode.

        It translates some of Django's test command option to pytest's.
        """
        import pytest

        argv = []
        if self.verbosity == 0:
            argv.append('--quiet')
        if self.verbosity == 2:
            argv.append('--verbose')
        if self.verbosity == 3:
            argv.append('-vv')
        if self.failfast:
            argv.append('--exitfirst')
        if self.keepdb:
            argv.append('--reuse-db')

        argv.extend(test_labels)
        return pytest.main(argv)

Add the path to this class in your Django settings:

TEST_RUNNER = 'my_project.runner.PytestTestRunner'

and invoke the tests via

$ python manage.py test <django args> -- <pytest args>

However, you won't be able to run the tests via direct invocation of pytest with this solution.