Search code examples
djangopostgresqldjango-southdjango-migrations

Django migrations failing with fresh database

wondering if anyone else is experiencing frustrating behavior with Django migrations. I have looked at several related posts on SO but have not found anything addressing this specifically.

This keeps happening to me, and it MUST be a common scenario, so what am I missing?

My scenario:

  1. List item
  2. I've built a Django application, iterating using sqlite on my local machine
  3. I now want to deploy to a production Postgresql db (AWS or elsewhere)
  4. I set up my new database, add it to my settings, and run python manage.py migrate
  5. I get a SQL error telling me that one of my model tables doesn't exist. Duh! That's why I'm running migrate.
  6. I end up having to hand-make tables, do weird things like python manage.py migrate <app_name> --fake which creates further issues with packages like simple_history

I'm sure there must be an obvious answer -- what should I do to set up database schema on a brand new, clean database?

EDIT: Traceback of error I am getting:

When I run python manage.py migrate on a blank database, I get the following error:

(banzai-platform-django) ➜  src git:(master) ✗ ./manage.py migrate
Traceback (most recent call last):
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: relation "events_event" does not exist
LINE 1: ...ents_event"."created", "events_event"."slug" FROM "events_ev...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
    utility.execute()
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/management/__init__.py", line 355, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/management/base.py", line 327, in execute
    self.check()
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/management/base.py", line 359, in check
    include_deployment_checks=include_deployment_checks,
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 62, in _run_checks
    issues.extend(super(Command, self)._run_checks(**kwargs))
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/management/base.py", line 346, in _run_checks
    return checks.run_checks(**kwargs)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/checks/registry.py", line 81, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/checks/urls.py", line 16, in check_url_config
    return check_resolver(resolver)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/core/checks/urls.py", line 26, in check_resolver
    return check_method()
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/urls/resolvers.py", line 254, in check
    for pattern in self.url_patterns:
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/urls/resolvers.py", line 405, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/urls/resolvers.py", line 398, in urlconf_module
    return import_module(self.urlconf_name)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/src/banzai_platform/urls.py", line 67, in <module>
    url(r'^registrations/', include('registrations.urls')),
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/conf/urls/__init__.py", line 50, in include
    urlconf_module = import_module(urlconf_module)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/src/registrations/urls.py", line 3, in <module>
    from .views import (
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/src/registrations/views.py", line 160, in <module>
    class ConfirmEventListView(ListView):
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/src/registrations/views.py", line 164, in ConfirmEventListView
    queryset = [obj for obj in events if obj.need_confirmation > 0]
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/models/query.py", line 250, in __iter__
    self._fetch_all()
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
    raise original_exception
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
    cursor.execute(sql, params)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/backends/utils.py", line 80, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/joedavy/Dropbox/Apps/banzai-platform-django/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "events_event" does not exist
LINE 1: ...ents_event"."created", "events_event"."slug" FROM "events_ev...
Solution

  • Your traceback shows that you are running a list comprehension over a queryset at the class level inside your ConfirmEventListView. Anything at class level is executed when the class is first imported; so this query is being done immediately any Django command is executed, which is before the migrations have had a chance to run.

    You must never do anything that accesses the database at this level. (To be honest you shouldn't be doing this list comprehension at all, since it is returning a list not a queryset; so it won't work with pagination or other things that expect an actual queryset. However, since the rest of your code works, we can assume that this isn't a problem in your case - yet.)

    If need_confirmation is a field, your queryset should just be defined as Event.objects.filter(need_confirmation__gt=0). However, if it's a model method, then you should define a get_queryset() method on the ConfirmEventListView and do this processing there, instead of at class level.