Python 3.x: How should one override inherited properties from parent classes?

Question

A simple example probably shows more:

class NaturalNumber():
    def __init__(self, val):
        self._value = val

    def _get_value(self):
        return self._value

    def _set_value(self, val):
        if val < 0:
            raise ValueError(
                f"Cannot set value to {val}: Natural numbers are not negative."
            )
        self._value = val

    value = property(_get_value, _set_value, None, None)


class EvenNaturalNumber(NaturalNumber):
    # This seems superfluous but is required to define the property.
    def _get_value(self):
        return super()._get_value()

    def _set_value(self, val):
        if val % 2:
            raise ValueError(
                f"Cannot set value to {val}: Even numbers are divisible by 2."
            )
        super()._set_value(val)

    # This seems superfluous but parent property defined with parent setter.
    value = property(_get_value, _set_value, None, None)

This is a simplified example from a real usecase where I want to inject code for testing over production classes. The principle is the same, and here I am introducing an extra validation to value in the EvenNaturalNumber class that inherits from the NaturalNumber class. My boss doesn't like me getting rid of all his decorators, so ideally a solution should work however the underlying class is written.

What would seem natural is:

class NaturalNumber():
    def __init__(self, val):
        self._value = val
    
    @property
    def value(self):
        return self._value
    
    @value.setter
    def value(self, val):
        if val < 0:
            raise ValueError(
                f"Cannot set value to {val}: Natural numbers are not negative."
            )
        self._value = val


class EvenNaturalNumber(NaturalNumber):
    @property
    def value(self):
        return super().value
    
    @value.setter
    def value(self, val):
        if val % 2:
            raise ValueError(
                f"Cannot set value to {val}: Even numbers are divisible by 2."
            )
        super().value = val

But this errors with valid sets on EvenNaturalNumber.value (say, = 2). With AttributeError: 'super' object has no attribute 'value'.

Personally, I would say that's a bug in the python language! But I'm suspecting I have missed something.

---

I have found a solution using decorators but this seems rather convoluted:

class NaturalNumber():
    def __init__(self, val):
        self._value = val

    @property
    def value(self):
        return self._value

    @value.setter
    def value(self, val):
        if val < 0:
            raise ValueError(
                f"Cannot set value to {val}: Natural numbers are not negative."
            )
        self._value = val


class EvenNaturalNumber(NaturalNumber):
    @property
    def value(self):
        return super().value
    
    @value.setter
    def value(self, val):
        if val % 2:
            raise ValueError(
                f"Cannot set value to {val}: Even numbers are divisible by 2."
            )
        super(type(self), type(self)).value.fset(self, val)

And another way is:

class NaturalNumber():
    def __init__(self, val):
        self._value = val
    
    @property
    def value(self):
        return self._value
    
    @value.setter
    def value(self, val):
        if val < 0:
            raise ValueError(
                f"Cannot set value to {val}: Natural numbers are not negative."
            )
        self._value = val


class EvenNaturalNumber(NaturalNumber):
    # This seems superfluous but is required to define the property.
    def _set_value(self, val):
        if val % 2:
            raise ValueError(
                f"Cannot set value to {val}: Even numbers are divisible by 2."
            )
        NaturalNumber.value.fset(self, val)

    # This seems superfluous as parent property defined with parent setter.
    value = NaturalNumber.value.setter(_set_value)

But this second solution seems rather unsatisfactory as it makes use of the knowledge that the value property is defined in the NaturalNumber class. I don't see any way to iterate over the EvenNaturalNumber.__mro__ unless I do this in EvenNaturalNumber._set_value(self, val), but that's the job of super(), isn't it?

Any improvement or suggestions will be gratefully received. Otherwise, my boss is just going to have to live with super(type(self), type(self)).value.fset(self, val)!

---

Added 01/01/2024.

Many thanks to ShadowRanger for pointing out the error with super(type(self), type(self)); I completely agree this should be super(__class__, type(self)) (mia culpa).

Why this has arisen is as follows. My parent classes would be better named Device (equivalent to NaturalNumber above). These talk to external devices over a serial connection (using pyserial). Various attributes on the external devices are exposed in the class as properties. So a getter will write to the external device to query a value and return the reply as the value appropriately decoded and typed for the callee. The setter writes to the device to set the value there and then checks there has been an appropriate reponse from the device (all with apropriate error handling for exceptional circumstances). Needless to say, the whole system is about getting these devices to interact in interesting ways. All of this seems quite natural.

I am building a test system over the top of this so that (to some extent) the entire system can be tested (unit / functional / regression etc.) without actually being connected to any of the devices: I hesitate to use the phrase "test harness" but perhaps justified in this case. So the idea is to use inheritance over each Device class and have a MockDevice class that inherits from its parent and exposes the same properties. But each MockDevice class is initialised with a MockSerialConnection (as opposed to a real SerialConnection which the Device classes are initialised with), so that we can inject the expected responses into the MockSerialConnection, which are then read by the Device code and (hopefully) interpreted correctly, thus providing a mechanism to test changes to the code as the system develops. It is all smoke and mirrors, but hopefully in a good way.

For the MockDevice properties the getters and setters need to set the relevant serial communication and then call the relevant getters and setters of the parent Device class, so that we have good code coverage. There is quite a lot of multiple inheritance going on here too (thank goodness for the __mro__), and exactly where in the inheritance hierachy a method or property is defined isn't completely fixed (and may vary in the future). We start with AbstractDevice classes that essentially define functionality of the device (with a whole class heirachy here for similar devices with more or fewer features or functionality), the actual Device class represents a specific device (down to catalogue number) from a given manufacturer, which is not only dependant of the AbstractDevice but the communication protocol (and type of serial connection) together with specifics (such as the commands to send for a specific attribute).

The helper function solution while good for providing functionality that is easily overriden (library authors take note) doesn't quite fit the bill here for the purpose of testing. There is nothing stopping another developer down the road applying a "quick fix" to the setters and getters (rather than the relevant helper functions), that would never be picked up by the test system. I also don't really see the difference from my first code sample where I specifically define _get_value and _set_value and declare the property with value = property(_get_value, _set_value, None, None) and avoid the decorators.

I am also a little disatisfied with the @NaturalNumber.value.setter solution because it needs apriori knowledge that the property being set resides in the NaturalNumber class (the improvement over my earlier solution accepted). Yes, I can work it out how it is now but as a test system it should work if down the line a developer moves the functionality to another place in the inheritance hierarchy. The test system will still error but it means we will need to maintain code in two places (production and test); it seems preferable that the test system walks the mro to find the relevant class property to override. If one could @super().value.setter then that would be perfect, but one can't.

Notice that there is no problem in the getters:

class EvenNaturalNumber(NaturalNumber):
    @property
    def value(self):
        return super().value

works just fine. This is why I am tempted to think of this as a "language bug". The fact that super(__class__, type(self)).value.fset(self, val) works suggests (to me) that the python compiler would be able to detect super().value = val as a setter and act accordingly. I would be interested to know what others thought about this as a Python Enhancement Proposal (PEP), and if supportive what else should be addressed around the area of properties. I hope that helps to give a fuller picture.

Also if there are other suggestions on a different way to approach the test harness, that will also be gratefully received.

Answer 1

So first off:

super(type(self), type(self)) is straight up wrong; never do that (it seems like it works when there is one layer, it fails with infinite recursion if you make a child class and try to invoke the setter on it).

Sadly, this is a case where there is no elegant solution. The closest you can get, in the general case, is the safe version of super(type(self), type(self)), using that code as-is, but replacing super(type(self), type(self)) with super(__class__, type(self)). You can simplify the code a little by having the child class use a decorator rather than manually invoking .setter, getting the benefits of your value = NaturalNumber.value.setter(_set_value) solution more succinctly:

class EvenNaturalNumber(NaturalNumber):
    # No need to redefine the getter, since it works as is
    # Just use NaturalNumber's value for the decorator
    @NaturalNumber.value.setter
    def value(self, val):
        if val % 2:
            raise ValueError(
                f"Cannot set value to {val}: Even numbers are divisible by 2."
            )
        # Use __class__, not type(self) for first argument,
        # to avoid infinite recursion if this class is subclassed
        super(__class__, type(self)).value.fset(self, val)

__class__ is what no-arg super() uses to get the definition-time class for the method (whenever super or __class__ are referenced in a function being defined within a class, it's given a faked closure-score that provides __class__ as the class the function was defined in). This avoids the infinite recursion problem (and is slightly more efficient as a side-benefit, since loading __class__ from closure scope is cheaper than calling type(self) a second time).

That said, in this particular case, the best solution is probably to do as DarkMath suggests, and use an internal validation method that the setter can depend on, so the property need not be overridden in the child at all. It's not a general solution, since the changes aren't always so easily factored out, but it's the best solution for this particular case.