I asked a question to try and fix an issue I was having where I didn't understand how to use ForeignKeys properly. That was very helpfully fixed, although I had a subsequent problem with django.db.utils.IntegrityError: core_team.blahblah_id may not be NULL and I decided to roll back, do something slightly simpler - in order to avoid a double lookup (Match is linked to both Team and League), I would write a management command to import the teams.
On my (clearly flawed) understanding from the previous question, I've done it right - it uses get_or_create to check for the league instance and then assigns the team based on that. I've also doublechecked that the DB is up to date (I'm running south and did the forward migration the last time I changed the scheme, nothing's changed since then. Last change was to make the names in both models the primary key (as there's only one team of each name, only one league of each name.)
Most recently, I've added code to provide the default to the team get_or_create section, but am receiving the same error. I understand the cause (I think) of the error - that the ForeignKey 'league' in Team already exists in the database, and can't be null inserting another team (from https://docs.djangoproject.com/en/1.5/ref/models/querysets/#get-or-create), just not how to fix it.
Management command:
from django.core.management.base import BaseCommand, CommandError
import csv
import csvImporter
#from core.models import Match
from time import strptime
from datetime import datetime
master_data = open ('/Users/chris/Desktop/AllDataTruncated.csv', 'r')
data = list(tuple(rec) for rec in csv.reader(master_data, delimiter=','))
from core.models import League, Team
team_list = []
for row in data:
if row[2] == "HomeTeam":
print "Continuing"
continue
elif row[2] == "":
print "Continuing"
continue
else:
league, _ = League.objects.get_or_create(name=row[0])
print league
team, _ = Team.objects.get_or_create(team_name=row[2], defaults={'league':league})
current_team = Team(league = league, team_name=team)
print current_team
And relevant bits of models.py:
class League (models.Model):
name = models.CharField(max_length=2, primary_key=True)
last_modified = models.CharField(max_length=50)
def __unicode__(self):
return unicode(self.name)
class Team(models.Model):
team_name = models.CharField(max_length=50, primary_key=True)
league = models.ForeignKey(League)
team_colour = models.CharField(max_length=6, null=True, blank=True)
def __unicode__(self):
return unicode (self.team_name)
The full traceback is:
$ python manage.py importteams
Continuing
E0
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
utility.execute()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 272, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 77, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/chris/Dropbox/Django/gmblnew/core/management/commands/importteams.py", line 26, in <module>
team2, _ = Team.objects.get_or_create(team_name=row[3])
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/manager.py", line 146, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 487, in get_or_create
six.reraise(*exc_info)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 477, in get_or_create
obj.save(force_insert=True, using=self.db)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/base.py", line 546, in save
force_update=force_update, update_fields=update_fields)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/base.py", line 650, in save_base
result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/manager.py", line 215, in _insert
return insert_query(self.model, objs, fields, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 1661, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 937, in execute_sql
cursor.execute(sql, params)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/util.py", line 41, in execute
return self.cursor.execute(sql, params)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 364, in execute
six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 362, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: core_team.league_id may not be NULL
Now - I think that it's odd that it's saying league_id here, as this shouldn't be relevant anymore? When I did the migration, the question came up:
? The field 'League.id' does not have a default specified, yet is NOT NULL.
? Since you are removing this field, you MUST specify a default
? value to use for existing rows. Would you like to:
? 1. Quit now.
? 2. Specify a one-off value to use for existing columns now
? 3. Disable the backwards migration by raising an exception; you can edit the migration to fix it later
? Please select a choice: 3
Is this what's perpetuating this issue?
Edit: Seems not. Dropped the DB and moved the South migrations folder, and it's still doing it. The source CSV is also fine (no blank lines or empty strings/segments), and the code above has a section to skip those segments anyway; it's not getting that far.
Ugh. The answer to this, for any other newbies who are coming to it later, is actually ludicrously simple. What I'm doing here is creating an entry in the table 'Team', which has a ForeignKey going back to 'League'.
The 'trick' (it's not a trick, just really badly explained in the documentation, imho) is that you need to explicitly pass the league back when you do the get_or_create for the Team object. It's not just about matching the team name
I thought I'd done this, but I hadn't, it appears. This code works (and quite effectively ensures there are no duplicates):
for row in data:
if row[2] == "HomeTeam":
print "Continuing"
continue
elif row[2] == "":
print "Continuing"
continue
else:
league, _ = League.objects.get_or_create(name=row[0])
print league
team, _ = Team.objects.get_or_create(team_name=row[2], league=league)
current_team = Team(league = league, team_name=team)
print current_team