Search code examples
mysqldatabasedjangopostgresqlbackup

Django using dumpdata to create database backups


I have a production server running, and a local development one (this one uses the simple runserver, etc.).

I was planning on using Django's built in "dumpdata" command in order to create backups of the database every so often. Sadly, I can't seem to get "loaddata" to take in what "dumpdata" provides it.

On the production server:

python manage.py dumpdata > db_backup.json

This gives me a ~6MB file, which I then transfer over to my local machine. On my local machine:

bash-3.2$ python manage.py flush --settings=config.settings_dev
/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/fields/subclassing.py:80: DeprecationWarning: A Field class whose get_db_prep_save method hasn't been updated to take a `connection` argument.
  new_class = super(SubfieldBase, cls).__new__(cls, name, bases, attrs)
You have requested a flush of the database.
This will IRREVERSIBLY DESTROY all data currently in the 'dcif_db' database,
and return each table to the state it was in after syncdb.
Are you sure you want to do this?

    Type 'yes' to continue, or 'no' to cancel: yes

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): no
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_manager(settings)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/commands/flush.py", line 74, in handle_noargs
    emit_post_sync_signal(set(all_models), verbosity, interactive, db)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/sql.py", line 190, in emit_post_sync_signal
    interactive=interactive, db=db)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/dispatch/dispatcher.py", line 172, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/contrib/auth/management/__init__.py", line 51, in create_permissions
    content_type=ctype
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/manager.py", line 138, in create
    return self.get_query_set().create(**kwargs)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/query.py", line 360, in create
    obj.save(force_insert=True, using=self.db)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/base.py", line 460, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/base.py", line 553, in save_base
    result = manager._insert(values, return_id=update_pk, using=using)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/manager.py", line 195, in _insert
    return insert_query(self.model, values, **kwargs)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/query.py", line 1436, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 791, in execute_sql
    cursor = super(SQLInsertCompiler, self).execute_sql(None)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
    cursor.execute(sql, params)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/backends/util.py", line 34, in execute
    return self.cursor.execute(sql, params)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: can't adapt type '__proxy__'

I've had this problem a few weeks ago, and just running whatever command generated the error a second time made the error go away. Not the ideal solution, but I'm not sure how else to fix it.

Then:

bash-3.2$ python manage.py loaddata etc/db_backups/12_9_11.json --settings=config.settings_dev
/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/fields/subclassing.py:80: DeprecationWarning: A Field class whose get_db_prep_save method hasn't been updated to take a `connection` argument.
  new_class = super(SubfieldBase, cls).__new__(cls, name, bases, attrs)
Problem installing fixture 'etc/db_backups/12_9_11.json': Traceback (most recent call last):
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 174, in handle
    obj.save(using=using)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
    models.Model.save_base(self.object, using=using, raw=True)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/base.py", line 570, in save_base
    created=(not record_exists), raw=raw, using=using)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/dispatch/dispatcher.py", line 172, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/Applications/djangostack-1.3-0/python/lib/python2.6/site-packages/djangobb_for_dcif/signals.py", line 18, in post_saved
    profile = post.user.forum_profile
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/fields/related.py", line 315, in __get__
    rel_obj = QuerySet(self.field.rel.to).using(db).get(**params)
  File "/Applications/djangostack-1.3-0/apps/django/lib/python2.6/site-packages/django/db/models/query.py", line 349, in get
    % self.model._meta.object_name)
DoesNotExist: User matching query does not exist.

Shouldn't the .json file provided by dumpdata be consistent, and not have missing objects as this error seems to claim?

On the production server, I'm using MySQL, and I'm using PostgreSQL locally, so I'm not sure if that's the cause of the problem.

Is there an easier way to do this?


Solution

  • I think best way to do this export tables as simple sql inserts statements then you can re rerun insert sql scripts in target db

    I'am using this method. I also tried dumpdata etc but insert statement way is easy and not complex.