How do the __slot__ descriptors work in python?

Question

I know what __slots__ does and what it's supposed to be used for.

However I have not found a comprehensive answer as to how the underlaying mechanisms of the member descriptor created using __slots__ works.

Where are the object-level values actually stored?

Is there a way to change these values without direct attribute access to the descriptors?
(ex. when class C has __dict__ you can do C.__dict__['key'] instead of C.key)

Can one "extend" the immutability of an object defining __slots__ by creating similar class-level descriptors? And as further elaboration of this; can one build an immutable object using metaclasses but not defining __slots__ explicitly by creating said descriptors manually?

Answer 1

__slot__ attributes are allocated in the native memory representation of the object, and the descriptor associated to the class that access then actually uses native C methods in CPython to set and retrieve a reference to the Python objects attributed to each slot attribute on the class instances as a C structure.

The descriptor for slots, presented in Python with the name member_descriptor is defined here: https://github.com/python/cpython/blob/master/Objects/descrobject.c

You can't perform or enhance these descriptors from pure Python code in anyway without using CTypes to interact with native code.

It is possible to get to their type by doing something like

class A:
   __slots__ = "a"

member_descriptor = type(A.a)

And then one could supose it would be possible to inherit from it, and write derived __get__ and __set__ methods that could do chekings and such - but unfortunately, it won't work as a base class.

However, it is possible to write other, parallel, descriptors that could in turn call the native descriptors to actually store the values. By using a metaclass, it is possible at class creation time to rename the passed in __slots__ and wrap their access in custom descriptors that could perform extra checks - and even hide then from "dir".

So, for a naive type checking slots variant metaclass, one could have

class TypedSlot:
    def __init__(self, name, type_):
        self.name = name
        self.type = type_

    def __get__(self, instance, owner):
        if not instance:
            return self
        return getattr(instance, "_" + self.name)

    def __set__(self, instance, value):
        if not isinstance(value, self.type):
            raise TypeError
        setattr(instance, "_" + self.name, value)


class M(type):
    def __new__(metacls, name, bases, namespace):
        new_slots = []
        for key, type_ in namespace.get("__slots__", {}).items():
            namespace[key] = TypedSlot(key, type_)
            new_slots.append("_" + key)
        namespace["__slots__"] = new_slots
        return super().__new__(metacls, name, bases, namespace)

    def __dir__(cls):
        return [name for name in super().__dir__() if  name not in cls.__slots__]