Overwriting __class__ raises TypeError

Question

I'm trying to extend functionality of a module class written in C by deriving from that class and overwriting / adding certain methods*.

Problem is that his module creates instances of the class I'm trying to extend at various different places. I thus need to create a casting method to cast the base class provided by the module to my own derived class with the additonal functionality.

Here's what I've tried:

class Derived(Base):
    @classmethod
    def cast(cls, obj: Base):
        obj.__class__ = cls

This method works with class hierarchies that I've created myself - however it fails with the module I use, throwing the following exception:

TypeError: __class__ assignment only supported for heap types or ModuleType subclasses

I'm having a hard time finding official information about this exception. Any info helps, I'd even accept hacky solutions so long as they're clean and fully mimic the behavior I'm looking for.

* The class I'm trying to extend is Surface inside the package pygame.

Answer 1

The __class__ attribute has always been restricted in what is acceptable, and Python classes can be a lot more flexible than types defined in C are.

For example, the __slots__ section in the datamodel documentation states:

__class__ assignment works only if both classes have the same __slots__.

and the same document calls the instance.__class__ attribute read only:

The implementation adds a few special read-only attributes to several object types, where they are relevant.

So __class__ was actually not meant to be writeable, but it has been for a long time and is a very useful property.

Now, in your case the assignment isn't permitted because the target instances are of a type that doesn't allocate the objects on a heap (an area of process memory that grows to accommodate an arbitrary number of objects, it's where most Python objects are allocated). Objects not allocated on the heap are managed differently (not subject to reference counting for example), and if you changed their type they'd suddenly need to be managed differently. That's currently not supported.

It was briefly supported in Python 3.5 release candidate builds, to allow setting __class__ on modules, but that backfired once it was discovered that it was possible to alter the type of interned immutable values:

class MyInt(int):
    # lets make ints mutable!

(1).__class__ = MyInt

1 is an interned value, so now all use of the integer 1 everywhere in the Python program has been changed. Not what you want, especially if you are re-using interpreter memory across multiple processes the way the Google App Engine does! See issue #24912 for the low-down.

But this is why module instances are specifically mentioned in the exception, see Customising module attribute access.

You'll have to find a different path to solving your problem. For example, perhaps your specific problem can be solved instead by wrapping the instances in something that uses __getattr__ to proxy attributes to the wrapped instance.