I have a postgres DB that looks like this:
Table "public.available_dates"
Column | Type | Modifiers | Storage | Stats target | Description
------------+-----------------------------+--------------------------------------------------------------+---------+--------------+-------------
id | integer | not null default nextval('available_dates_id_seq'::regclass) | plain | |
unix_day | integer | | plain | |
hour | integer | | plain | |
created_at | timestamp without time zone | | plain | |
updated_at | timestamp without time zone | | plain | |
Indexes:
"available_dates_pkey" PRIMARY KEY, btree (id)
"index_available_dates_on_unix_day_and_hour" UNIQUE, btree (unix_day, hour)
"index_available_dates_on_unix_day" btree (unix_day)
Has OIDs: no
This is hooked into a rails app I'm developing and a python app that runs as a service, I use peewee as the orm for python. The rails app runs fine, and I need each of the indexes on this db.
The python application looks at files, and adds dates to the db. The problem I am having is if the date+hour exists, it fails and ends the python app. I don't want dupes in the db, but I also want the app to continue running if this integrity error is thrown. This is the code I have so far:
data_source = [{'unix_day': 1370044800, 'created_at': datetime.datetime(2014, 7, 14, 10, 12, 57, 488000), 'updated_at': datetime.datetime(2014, 7, 14, 10, 12, 57, 488000), 'hour': 1},
...
]
try:
with db.transaction():
Available_Dates.insert_many(data_source).execute()
except IntegrityError as e:
print e
db.rollback()
pass
else:
db.commit()
db.close()
Which fails (as it should):
duplicate key value violates unique constraint "index_available_dates_on_unix_day_and_hour" DETAIL: Key (unix_day, hour)=(1370044800, 10) already exists.
How do I get my code to say, "That failed? Oh well... I just try the next one."
Note: I expect it will fail more than it will succeed.
Edit: Option 1 as per first answer:
for data in data_source:
try:
av_date = Available_Dates()
av_date.unix_day = data['unix_day']
av_date.hour = data['hour']
av_date.created_at = data['created_at']
av_date.updated_at = data['updated_at']
av_date.save()
except Exception as e:
pass
This seems not great as I'm catching all exceptions... Also, I have to assign every member separately which seems inefficient. It also fails because there is a lock on the db.
duplicate key value violates unique constraint "index_available_dates_on_unix_day_and_hour"
DETAIL: Key (unix_day, hour)=(1370044800, 1) already exists.
current transaction is aborted, commands ignored until end of transaction block
current transaction is aborted, commands ignored until end of transaction block
current transaction is aborted, commands ignored until end of transaction block
current transaction is aborted, commands ignored until end of transaction block
current transaction is aborted, commands ignored until end of transaction block
...
Here is what I did (thanks to ZJS):
for data in data_source:
try:
exists = Available_Dates.get(Available_Dates.unix_day == data['unix_day'],
Available_Dates.hour == data['hour'])
except Available_Dates.DoesNotExist:
av_date = Available_Dates.create(**data)
print av_date.id, "added to the database."
except Exception as e:
print e, type(e)
pass