Django-nonrel: Only part of fixture data loading in production

Question

As a learning project, I am using django-nonrel with the GAE to design a site with as much data as possible about the NBA. As such, I need to prepopulate my database (I'm using the GAE-datastore) with most of the data using a fixture, as entering info for 400+ players by hand is simply unthinkable. To that end I've created a .yaml fixture with the following format (taken straight from the django docs example):

- model: players.player
  pk: 1
  fields: 
    team:  10
    first_name:  Jeff
    last_name:  Adrien
    age:  25
    pos:  SF
    gp:  8

- model: players.player
  pk: 2
  fields: 
    team:  7
    first_name:  Arron
    last_name:  Afflalo
    age:  26
    pos:  SG
    gp:  32

Unfortunately, when I run "python manage.py remote loaddata nbadata.yaml', it only loads around the first 190 players (I say around because the first time it loaded 190, the second time it loaded 186) before crapping out and giving me the following error message:

File "/usr/local/google_appengine/lib/fancy_urllib/fancy_urllib/__init__.py", 
line 367, in do_open
raise url_error
URLError: <urlopen error [Errno 101] Network is unreachable>

Does anyone know what's going on, and whether there is a solution?

UPDATE: I changed the name of my .yaml file to 'initial_data.yaml' to see if the automatic load would fare better during the syncdb portion of the 'python manage.py deploy' process. I got the following result:

Running syncdb.
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 448 object(s) from 1 fixture(s)

However, when I look in the datastore, there are still only 186 players! Does anyone know what's going on??

Answer 1

The remote API makes an HTTP request to upload each HTTP entity. At some point, your network connection is failing, and the entire operation fails.

Can think of two solutions, but neither is particularly easy: - Find a better internet connection. - Patch the fixture upload code in django-nonrel to do some automatic retrying instead of failing.

I'm not certain, but I don't think syncdb really works on production, I believe it just runs against the local datastore, that's why you see that succeeding.