Search code examples
pythonpython-2.7copygetattr

Python deepcopy with custom __getattr__ and __setattr__

I have implemented a class that can distinguish between a fixed set of instance attributes (let's call them meta-attributes) and an arbitrary set of other instance attributes.

It has custom __getattr__ and __setattr__:

class MyClass(object):

    def __init__(self, meta1, meta2, **other_attr):
        super(MyClass, self).__setattr__('meta1', meta1)
        super(MyClass, self).__setattr__('meta2', meta2)
        super(MyClass, self).__setattr__('params', {})

        self.params = {key: other_attr[key] for key in other_attr}


    # this is called when default lookup finds nothing
    def __getattr__(self, key):
        print('__getattr__({})'.format(key))
        try:
            return self.params[key]
        except KeyError:
            raise AttributeError(key)


    # this is called always
    def __setattr__(self, key, value):
        print('__setattr__({}, {})'.format(key, value))
        if key in self.__dict__:
            super(MyClass, self).__setattr__(key, value)
        else:
            self.params[key] = value

This works fine, all meta-attributes go directly into the __dict__ of an instance, while all the other attributes go into the params dictionary:

obj1 = MyClass(meta1 = 'foo', meta2 = 'bar', x=1, y=2, z=3)
obj1.w = 4
print(obj1.__dict__)

Output:

__setattr__(params, {'y': 2, 'x': 1, 'z': 3})
__setattr__(w, 4)
{'meta1': 'foo', 'meta2': 'bar', 'params': {'y': 2, 'x': 1, 'z': 3, 'w': 4}}

Except when I try to deepcopy my object, it does something strange:

import copy
obj1 = MyClass(meta1='foo', meta2='bar', x=1, y=2, z=3)
obj2 = copy.deepcopy(obj1)

Output:

__setattr__(params, {'y': 2, 'x': 1, 'z': 3})
__getattr__(__deepcopy__)
__getattr__(__getnewargs__)
__getattr__(__getstate__)
__getattr__(__setstate__)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
...
and then it calls __getattr__ about a hundred more times

In the end it does create a copy, but why does it make so many calls to __getattr__?

Solution

  • This is causing a recursive lookup (remember obj2 is not initialised via __init__)

    return self.params[key]

    You should instead do this

    return super().__getattribute__('params')[key]