What specific exceptions represent a serialization failure when Django is using serializable transaction isolation level with postgresql?

Question

Sometimes it's desirable to use a higher isolation level than the default "read committed" for database operations in Django. The docs warn that:

Under higher isolation levels, your application should be prepared to handle exceptions raised on serialization failures.

But which specific exceptions indicate a serialization failure, versus some other problem with the query or transaction?

A simple retry mechanism after a serialization failure might look like something like this:

for retries in range(0, 3):
    try:
        with transaction.atomic():
            MyModel.objects.update(foo='bar')
    except StuffHappened:
        continue
    else:
        break

What specific exceptions should replace StuffHappened so that only serialization failures, and not other exceptions, result in a retry?

Django has a variety of Database Exceptions and Transaction Exceptions. Might one/some of those represent serialization failures?

I'm specifically interested in postgresql for this.

Answer 1

Hmm, good question. The documentation implies that the appropriate exception would be a TransactionManagementError:

TransactionManagementError is raised for any and all problems related to database transactions.

However, the source code gives a strong clue that it isn't:

class TransactionManagementError(ProgrammingError):
    """Transaction management is used improperly."""
    pass

Note that this is a ProgrammingError, which is indeed used to indicate programmer error (i.e. "used improperly").

If we look at the documentation for psycopg (the Python adapter used for PostgreSQL support) we see that it will raise a psycopg2.extensions.TransactionRollbackError:

exception psycopg2.extensions.TransactionRollbackError (subclasses OperationalError)

Error causing transaction rollback (deadlocks, serialization failures, etc).

But what does Django do with that? Well, as documented here, it wraps the standard Python DB API 2.0 exceptions in Django equivalents, and sets the __cause__ attribute to the original exception. So the following is probably the most specific check you can make:

from django.db import OperationalError
from psycopg2.extensions import TransactionRollbackError

for retries in range(0, 3):
    try:
        with transaction.atomic():
            MyModel.objects.update(foo='bar')
    except OperationalError as e:
        if e.__cause__.__class__ == TransactionRollbackError:
            continue
        else:
            raise            
    else:
        break

Depending on the error details exposed by PostgreSQL (available via e.__cause__.diag) it may be possible to write an even more specific test.

Generally, though, the Python DB API 2.0 documentation states that OperationalError is indeed the correct exception type for transaction issues, so catching that hopefully would be a reasonably effective database-agnostic solution.