Shelve (or pickle) doesn't save the dict of objects correctly. It just looses the data

Question

I decided to create a little tracking list for personal needings. I created two main classes to store and process data. First one represents the subject and an exercises list. The second one represents each exercise from exercises list (mainly just two variables, all(whole) answers and correct(good) answers).

class Subject:
    def __init__(self, name):
        self.name = name
        self.exercises = []

    def add(self, exc):
        self.exercises.append(exc)

    # here is also "estimate" and __str__ methods, but they don't matter


class Exercise:
    def __init__(self, good=0, whole=20):
        self._good  = good
        self._whole = whole

    def modify(self, good, whole=20):
        self._good  = good
        self._whole = whole

    # here is also "estimate" and __str__ methods, but they don't matter

I defined a dictionary, filled it with Subject instances, transferred it to the shelve file and saved it.

with shelve.open("shelve_classes") as db:
    db.update(initiate())

Here's the representation(initiate state):

#Comma splices & Fused sentences (0.0%)
#0/20      0.0%
#0/20      0.0%
#0/20      0.0%
#0/20      0.0%
#0/20      0.0%

After that, I tried to reopen dumped file and update some values.

with shelve.open('shelve_classes') as db:
    key = 'Comma splices & Fused sentences'
    sub = db[key]
    sub.exercises[0].modify(18)
    db[key] = sub

Looks ok, let's review it:

print(db[key])

#Comma splices & Fused sentences (18.0%)
#18/20     90.0%
#0/20      0.0%
#0/20      0.0%
#0/20      0.0%
#0/20      0.0%

But when I close the file, next time I open it, it comes back to initiate state and all corrections losses. Even tried it with pickle, doesn't work there either. Can't figure out, why it's not saving the data.

Answer 1

The shelve module doesn't notice when you mutate an object, only when you assign it:

Because of Python semantics, a shelf cannot know when a mutable persistent-dictionary entry is modified. By default modified objects are written only when assigned to the shelf.

So it doesn't recognize that sub.exercises[0].modify(18) is an action that needs to be rewritten back to disk.

Try setting the writeback flag to True when opening the db. Then it will re-save the db when it closes, even if it didn't explicitly detect any changes.

with shelve.open('shelve_classes', writeback=True) as db:
    key = 'Comma splices & Fused sentences'
    sub = db[key]
    sub.exercises[0].modify(18)
    db[key] = sub