Search code examples
pythonshelve

Python shelve ignoring writeback=True


I have a problem where changes in a list are not reflected in the associated shelf (Python 2.6.9). Consider the following minimal example

File Context.py:

import shelve

class Cont:
    shelf = shelve.open( STATUSFILE, protocol = 0, writeback = True )

File test.py:

from Context import Cont

class T:
    jobQueue = list()

    @classmethod
    def save(cls):
        cls.jobQueue = [1,2,3]
        Cont.shelf['queue'] = cls.jobQueue
        Cont.shelf.sync()
        print(Cont.shelf['queue'])
        cls.jobQueue.pop()
        print(Cont.shelf['queue'])
        print(id(cls.jobQueue))
        print(id(Cont.shelf['queue']))

Output:

[1,2,3]
[1,2,3]
7804904
7899472

First, I assumed the second list output to be[1,2]. Second, why are the ids different? When assigning the list, only a reference should be copied, keeping the id intact.
The strange thing is that I cannot reproduce that on the Python shell. There the output ids are the same and the shelfed list shows the change on jobQueue.
It seems that when executing my program the writeback=True is ignored. I am greatful for every hint!

EDIT: I did reproduce the problem on the shell with the given minimal example. May this have something to do with the class structure (I'm pretty new to objective Python)? I can imagine that class Cont does not know class T and hence cannot store a reference to T.jobQueue but rather a copy of the list.


Solution

  • Without shelf.sync():

    def save(cls):
        Cont.shelf['queue'] = cls.jobQueue   #1
        print(Cont.shelf['queue'])           #2
        # [1, 2, 3]        
        cls.jobQueue.pop()                   #3
        print(Cont.shelf['queue'])           #4
        # [1, 2]
    
    1. With writeback=True, assignments store the key/value pair in shelf.cache as well as shelf.dict.
    2. Attempts to retrieve the data at key 'queue' from shelf.cache.
    3. modifies cls.jobQueue, which is the same object as the one retrieved from the cache
    4. retrieves the data at key 'queue' from shelf.cache again. Since the cache holds a refence to cls.jobQueue, this is the same object.

    However, if you call shelf.sync():

    def save(cls):
        Cont.shelf['queue'] = cls.jobQueue   
        Cont.shelf.sync()                    #1
        print(Cont.shelf['queue'])           #2
        # [1, 2, 3]        
        cls.jobQueue.pop()                   #3
        print(Cont.shelf['queue'])           #4
        # [1, 2, 3]
    
    1. The shelve file is updated, and the cache is reset to an empty dict.
    2. Attempts to retrieve the data at key 'queue' from shelf.cache. Since the cache is empty, a new copy of the data is retrieved from the shelve file.
    3. modifies cls.jobQueue, which is not the same object as the copy just retrieved
    4. The cache is still empty, so this retrieves a new copy from the unupdated shelve file again