Search code examples
pythonslotspython-descriptors

Why the introduction of __slots__ was made possible with descriptors?

In this blog post from the series "The History of Python", Guido van Rossum states:

Another enhancement made possible with descriptors was the introduction of the __slots__ attribute on classes.

I understand this sentence as: under the hood __slots__ is implemented by descriptors.

But contrary to this my interpretation, Guido van Rossum writes a few lines later:

Underneath the covers, the implementation of this feature is done entirely in C and is highly efficient.

So, __slots__ is not implemented by descriptors?

But two sentences later, again he writes:

Not only was __slots__ an interesting application of descriptors, ...

So, what's actually the case with __slots__ and descriptors?

Is __slots__ implemented by descriptors or not? And if yes: how?

Solution

  • The statements don't contradict themselves. The attributes defined by __slots__ are descriptors on the created class and the implementation of that descriptor is written in C (assuming CPython).

    The descriptor class is called member_descriptor as can be seen by this example code:

    import inspect

class Test:
    __slots__ = 'a',
    def __init__(self, a):
        self.a = a

type(Test.a)                        # member_descriptor
inspect.isdatadescriptor(Test.a)    # True
inspect.ismemberdescriptor(Test.a)  # True

    And a quick search on the CPython repository on GitHub revealed the C implementation of it (Link for CPython version 3.8.0).

    To go into a bit more detail:

    A Python class is essentially a dict with (a lot of) bells and whistles. On the other hand there are Python-C-classes that use a C-struct to implement a Python class. Such a C-struct is faster and requires (significantly) less memory than a dictionary even if it only contains Python objects (which is basically a C-array containing references to Python objects).

    To make it possible that "normal" Python classes could benefit from the faster access and the reduced memory footprint __slots__ was introduced. A class with __slots__ will be translated essentially to a C-struct. However to make it possible that the attribute lookup/setting/deletion map to the corresponding struct member some sort of translation layer (descriptors) is required. That translation layer for members defined in __slots__ is member_descriptor.

    So when you look up the attribute on an instance of a __slots__-class you'll get a member_descriptor and that member_descriptor will know how to get/set/delete the member of the underlying C-struct.