Search code examples
pythonshelve

Bizarre behavior trying to store a deque in a Shelve


I will let the following terminal session speak for itself:

>>> import shelve
>>> s = shelve.open('TestShelve')
>>> from collections import deque
>>> s['store'] = deque()
>>> d = s['store']
>>> print s['store']
deque([])
>>> print d
deque([])
>>> s['store'].appendleft('Teststr')
>>> d.appendleft('Teststr')
>>> print s['store']
deque([])
>>> print d
deque(['Teststr'])

Shouldn't d and s['store'] point to the same object? Why does appendleft work on d but not on s['store']?


Solution

  • It turns out they're not the same so any operations you perform on them won't match:

    >>> import shelve
    >>> s = shelve.open('TestShelve')
    >>> from collections import deque
    >>> s['store'] = deque()
    >>> d = s['store']
    >>> id(s['store'])
    27439296
    >>> id(d)
    27439184
    

    To modify items as you coded, you need to pass the parameter writeback=True:

    s = shelve.open('TestShelve', writeback=True)
    

    See the documentation:

    If the writeback parameter is True, the object will hold a cache of all entries accessed and write them back to the dict at sync and close times. This allows natural operations on mutable entries, but can consume much more memory and make sync and close take a long time.

    You can also do it with writeback=False but then you need to write the code exactly as in the provided example:

    # having opened d without writeback=True, you need to code carefully:
    temp = d['xx']      # extracts the copy
    temp.append(5)      # mutates the copy
    d['xx'] = temp      # stores the copy right back, to persist it