Search code examples
pythonpickle

pickle module seems to watch the original object


Writing this code:

>>> class T:
>>>     version = '1.0'
>>> import pickle
>>> pickled = pickle.dumps(T)
>>> T.version = '2.0'
>>> PT = pickle.loads(pickled)
>>> PT.version
>>> '2.0'

Why does that happen? Should not be PT.version == '1.0' as it was when pickled? On the other hand, I'm seeing that

 >>> T
 <class '__main__.T'>
 >>> PT
 <class '__main__.T'>
 >>> id(PT) == id(T)
 True 

Are python class objects (not class instances, the class object itself) singletons or something like that? I would expect that there would be two different classes now but there seems to be just one, and two different aliases or references or names.


Solution

  • Functions and classes are both essentially pickled as references to a global variable, note, this is why you cannot pickle lambdas, or classes and functions not defined at the top level. From the docs

    Note that functions (built-in and user-defined) are pickled by “fully qualified” name reference, not by value. 2 This means that only the function name is pickled, along with the name of the module the function is defined in. Neither the function’s code, nor any of its function attributes are pickled. Thus the defining module must be importable in the unpickling environment, and the module must contain the named object, otherwise an exception will be raised. 3

    Similarly, classes are pickled by named reference, so the same restrictions in the unpickling environment apply. Note that none of the class’s code or data is pickled

    Note, the docs also provide an example of how to override this behavior for a class object:

    import io
    import pickle
    
    class MyClass:
        my_attribute = 1
    
    class MyPickler(pickle.Pickler):
        def reducer_override(self, obj):
            """Custom reducer for MyClass."""
            if getattr(obj, "__name__", None) == "MyClass":
                return type, (obj.__name__, obj.__bases__,
                              {'my_attribute': obj.my_attribute})
            else:
                # For any other object, fallback to usual reduction
                return NotImplemented
    
    f = io.BytesIO()
    p = MyPickler(f)
    p.dump(MyClass)
    
    del MyClass
    
    unpickled_class = pickle.loads(f.getvalue())
    
    assert isinstance(unpickled_class, type)
    assert unpickled_class.__name__ == "MyClass"
    assert unpickled_class.my_attribute == 1