Why does Django drop the SQL DEFAULT constraint when adding a new column?

Question

In the latest Django (2.2), when I add a new field to a model like this:

new_field= models.BooleanField(default=False)

Django runs the following commands for MySQL:

ALTER TABLE `app_mymodel` ADD COLUMN `new_field` bool DEFAULT b'0' NOT NULL;
ALTER TABLE `app_mymodel` ALTER COLUMN `new_field` DROP DEFAULT;
COMMIT;

While this works when everything is updated, this is very problematic because old versions of the application can no longer create models after this migration is run (they do not know about new_field). Why not just keep the DEFAULTconstraint?

Answer 1

Why not just keep the DEFAULT constraint?

Because Django handles the default model field option at the application level, not the database level. So the real question is why it sets the DEFAULT constraint at all.

On the first point, historically Django did not support database-level defaults for model fields. There was always some interest in changing that (the first issue on the subject is 18 years old), and certainly other frameworks (Rails, I think, and SQLAlchemy) showed that it was possible.

Recently, Django has added that option with a separate field option called db_default. The behavior of default remains the same, though, and there are good reasons beyond backwards compatibility for handling defaults at the application level. Such as: the ability to express arbitrarily complex computations; not having to worry about subtle incompatibilities across database engines; the ability to instantiate a new instance in code and have immediate access to the default value; the ability to present the default value to users in forms; and more.

Now, adding a new non-nullable field to an existing database is a very different use case. In that situation, you have to provide a default to the database for it to perform the operation. makemigrations will try to infer the right value from your default option if it can, and if not it will force you to specify a value from the command line. So the DEFAULT modifier is used for this limited purpose and then removed.

As you've noticed, the lack of database-level defaults in Django can make continuous deployment harder. But the solution is fairly straightforward: just re-add the default yourself in a migration. One of the great benefits of the migrations system is that it makes it easy to make arbitrary, repeatable, testable changes to your database outside of Django's ORM. So just add a new RunSQL migration operation:

operations = [
    # Add SQL for both forward and reverse operations
    migrations.RunSQL("ALTER TABLE app_mymodel ALTER COLUMN new_field SET DEFAULT 0;",
                      "ALTER TABLE app_mymodel ALTER COLUMN new_field DROP DEFAULT;")
]

You can put that in a new migration file or simply edit the automatically generated one. Depending on your database and its support for transactional DDL, the sequence of operations may or may not be atomic.