Generic lazy initializer class and magic methods

Question

I want to create a class that lazily initializes objects in the background when they are first used, but that behaves exactly like the initialized objects. It should be thread-safe. For example:

In[1]: l = LazyInitializer(list, (1, 2, 3, 4, 5))
In[2]: l.__len__()
5

My current implementation is this:

class LazyInitializer:

    def __init__(self, initializer, *args, **kwargs):
        self.__initializer = initializer
        self.__args = args
        self.__kwargs = kwargs

        self.__obj = None
        self.__lock = Lock()

    @property
    def _obj(self):
        with self.__lock:
            if self.__obj is None:
                self.__obj = self.__initializer(*self.__args, **self.__kwargs)

        return self.__obj

    def __getattr__(self, item):
        return getattr(self._obj, item)

This works for regular object members (functions and properties alike), but it does not for magic methods, e.g.,

In[2]: l.__len__()  # works
5
In[3]: len(l)  # doesn't work
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-e75269d816bd> in <module>
----> 1 len(l)

TypeError: object of type 'LazyInitializer' has no len()

An ad-hoc solution could be to explicitly implement all possible magic methods on LazyInitializer. However, isn't there any better way?

Answer 1

Indeed. Actually, I revisted this problem just this week. The problem is that for using the magic methods, often as a consequence of executing an operator, Python won't do "regular" attribute access: for performance reasons - which are baked into the language spec, and not just a detail of cPython, it will shortcut to check if the class or one of the superclasses have the magic method already in place (__getattr__ and __getattribute__ are not checked), and call it.

So, the only way to ensure any possible magic method is called in a transparent way is if your proxy lazy object has all the magic methods.

The only "proxy" that can work otherwise is "super" itself: but it is rather special, and a key component of the language. It might be possible that by studying the implementation of super in C you could come up with a similar transparent proxy (with some code in C), but it is no way guaranteed.

I once put a similar proxy that would offload any computation to a subprocess, so that I had a "transparent future object" - it is not pretty code, but I had used it even in production ocasionally:

https://bitbucket.org/jsbueno/lelo/src/master/