ZODB transaction commit calls getstate multiple times

I'm creating a caching system using ZODB. The objects I am putting in the DB required me to use the __getstate__ and __setstate__ methods because they contain images that I convert to strings and store in Blobs.

I didn't find anyone with a similar problem so I'm assuming I'm doing something stupid. The problem I'm having can be seen by running the following test code:

from ZODB import FileStorage, DB
from persistent.mapping import PersistentMapping
import transaction

class Test(object):
    def __init__(self, a):
        self.a = a

    def __getstate__(self):
        print "Entering getstate for %s" % self.a
        return self.__dict__.copy()

    def __setstate__(self, state):
        print "Entering setstate for %s" % state["a"]
        self.__dict__ = state

print "Creating tests..."
tests = [ Test(i) for i in range(3) ]

print "Connecting to database..."
storage = FileStorage.FileStorage("./test_db.fs", blob_dir="./test_blobs")
db = DB(storage)
conn = db.open()
root = conn.root()
root["cache"] = PersistentMapping()
transaction.commit()

print "Adding tests to db..."
for idx,t in enumerate(tests):
    print "Starting transaction of idx %d" % idx
    root["cache"][idx] = t
    transaction.commit()

Which results in:

Creating tests...
Connecting to database...
Adding tests to db...
Starting transaction of idx 0
Entering getstate for 0
Starting transaction of idx 1
Entering getstate for 0
Entering getstate for 1
Starting transaction of idx 2
Entering getstate for 0
Entering getstate for 1
Entering getstate for 2

As you can see from the output each transaction calls the getstate method for every previous transaction. Is this supposed to happen/what am I missing? Am I completely misunderstanding the use of transaction? Is it ok to use PersistentMappings like that?

I'm using ZODB 3.10.3 and Python 2.6.3.

P.S. I know normally you would just do the commit after doing all the operations, but the loop represents a series of possible method calls in a "cache" object with each one either adding or retrieving data from the ZODB.

Thanks for any help or understanding you can give.

Solution

After hours and hours of additional research and testing here's my answer to my own question...

The simple answer is you MUST subclass from persistent.Persistent.

I wanted to avoid doing this (and I though I could from some of the research I had done) because of my need for a custom __getstate__ and __setstate__ and I thought that inheriting Persistent I would have to do some extra work to get that to work correctly. It seems to work fine, although there might be problems if my objects that I'm storing changed later (but I'm not changing them once they are in the ZODB).

If anyone has any other advice or warnings on how I'm doing this it would be much appreciated. Thanks for anything you can provide.

ZODB transaction commit calls __getstate__ multiple times

ZODB transaction commit calls getstate multiple times