Whenever I try and run zobbpack I generate the error:
psycopg2.IntegrityError: null value in column "zoid" violates not-null constraint
Any ideas what is causing this and how to fix it?
relstorage 1.5.1, postgres 8, plone 4.2.1.1
2012-12-03 13:18:03,485 [zodbpack] INFO Opening storage (RelStorageFactory)...
2012-12-03 13:18:03,525 [zodbpack] INFO Packing storage (RelStorageFactory).
2012-12-03 13:18:03,533 [relstorage] INFO pack: beginning pre-pack
2012-12-03 13:18:03,533 [relstorage] INFO pack: analyzing transactions committed Mon Nov 26 12:31:54 2012 or before
2012-12-03 13:18:03,536 [relstorage.adapters.packundo] INFO pre_pack: start with gc enabled
2012-12-03 13:18:03,759 [relstorage.adapters.packundo] INFO analyzing references from objects in 97907 new transaction(s)
2012-12-03 13:18:03,761 [relstorage.adapters.scriptrunner] WARNING script statement failed: '\n INSERT INTO object_refs_added (tid)\n VALUES (%(tid)s)\n '; parameters: {'tid': 0L}
2012-12-03 13:18:03,761 [relstorage.adapters.packundo] ERROR pre_pack: failed
Traceback (most recent call last):
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 486, in pre_pack
conn, cursor, pack_tid, get_references)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 580, in _pre_pack_with_gc
self.fill_object_refs(conn, cursor, get_references)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 387, in fill_object_refs
self._add_refs_for_tid(cursor, tid, get_references)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 459, in _add_refs_for_tid
self.runner.run_script_stmt(cursor, stmt, {'tid': tid})
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/scriptrunner.py", line 52, in run_script_stmt
cursor.execute(stmt, generic_params)
IntegrityError: null value in column "zoid" violates not-null constraint
Traceback (most recent call last):
File "zodbpack.py", line 86, in <module>
main()
File "zodbpack.py", line 78, in main
skip_prepack=options.reuse_prepack)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/storage.py", line 1114, in pack
adapter.packundo.pre_pack(tid_int, get_references)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 486, in pre_pack
conn, cursor, pack_tid, get_references)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 580, in _pre_pack_with_gc
self.fill_object_refs(conn, cursor, get_references)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 387, in fill_object_refs
self._add_refs_for_tid(cursor, tid, get_references)
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/packundo.py", line 459, in _add_refs_for_tid
self.runner.run_script_stmt(cursor, stmt, {'tid': tid})
File "/usr/local/lib64/python2.6/site-packages/RelStorage-1.5.1-py2.6.egg/relstorage/adapters/scriptrunner.py", line 52, in run_script_stmt
cursor.execute(stmt, generic_params)
psycopg2.IntegrityError: null value in column "zoid" violates not-null constraint
Running zodbpack.py without GC enabled (thanks Martijn) worked and enabled the database to pack down from 3.8Gb to 887Mb.
So my zodbpack-conf.xml looks like:
<relstorage>
pack-gc false
<postgresql>
dsn dbname='zodb' user='user' host='host' password='password'
</postgresql>
</relstorage>
and i run it with:
python zodbpack.py -d 7 zodbpack-conf.xml
Note: after the pack completes you still need to vacuum the database to get the space back. I run this from the command line as:
psql zodb postgresuser
zodb=# SELECT pg_database_size('zodb');
zodb=# vacuum full;
zodb=# SELECT pg_database_size('zodb');
Interestingly, when i run the pack command from the ZMI Control Panel, I still get the same error as previously:
Site Error
An error was encountered while publishing this resource.
Error Type: IntegrityError
Error Value: null value in column "zoid" violates not-null constraint
So i am assuming the ZMI pack uses GC enabled, and that there is still an issue with null values in my site database...
Should I try and run some SQL to clean it out? If an object has no 'zoid' value then is it effectively inaccessible junk? A SQL example for this would be great too :)