Search code examples
pythondjangodjango-rest-frameworkpostgresql-11django-tests

Errors when running tests in parallel


Multiple issues arise when I try to run tests in parallel.

According to the docs, "test_" is prepended to the database name specified in DATABASES. I used the name "postgres", so the database created when running tests is called test_postgres. When running tests in parallel, the following databases are created (which is expected): test_postgres_1, test_postgres_2, test_postgres_3, and test_postgres_4. When running all tests with the --parallel=4 option, however, every test fails with the following message: django.db.utils.OperationalError: FATAL: database "postgres_x" does not exist where x can be 1, 2, 3 or 4. I can see that the following databases have been created: test_postgres_x where x can be 1, 2, 3 or 4. Where's "postgres_x" coming from? Why isn't "test_" being prepended to these?

Furthermore, if I manually create the expected databases postgres_x (x = 1 to 4), the migrations applied to the "main" database aren't applied to the clones. This results in errors like this: django.db.utils.ProgrammingError: relation "users_user" does not exist. Roughly 1/4 tests pass when using 4 cores.

Lastly, if I try to migrate postgres_x by using migrate --database=postgres_x, I get: django.db.utils.ConnectionDoesNotExist: The connection postgres_x doesn't exist.

I have ensured that all tests are isolated just so I can run them in parallel. What am I supposed to do?


Solution

  • Instead of building your test harness yourself I suggest using pytest and pytest-django and pytest-xdist this will handle the db creation and migration for each parallel worker. (pytest can run Django UnitTest tests without modification)