Python extension type: super() not finding method (a.k.a. attribute) in base class

Question

I am recoding PyCXX which is a C++ wrapper for Python.

The original (working) implementation for adding methods to a new style class involved creating an "extern C" handler function for each, populating a PyMethodDef table with pointers to these handlers, and placing the PyTypeObject's table->tp_methods to this table.

Instead I'm replacing the mechanism with one that overrides getattro, searches my own data to see if this attribute has a corresponding C++ method, if so packages it up into a Callable Python object and returns it, otherwise defers to PyObject_GenericGetAttr.

This technique works if I create an instance of new_style_class:

# new_style_class is a C++ class deriving from ExtObj_new
_new_style_class = simple.new_style_class()
_new_style_class.func_noargs()

However, if I attempt to derive from new style class and invoke func_noargs() from the base, like this:

print( '--- Derived func ---' )
class Derived(simple.new_style_class):
    def __init__( self ):
        simple.new_style_class.__init__( self )

    def derived_func( self ):
        print( 'derived_func' )
        print( vars(super()) )
        super().func_noargs() # <-- AttributeError: 'super' object has no attribute 'func_noargs'

d = Derived()
d.derived_func()

... it returns AttributeError: 'super' object has no attribute 'func_noargs'.

I'm wondering whether the problem comes from the fact that I am overriding getattro rather than getattr.

Is it possible that CPython, when it attempts to invoke a base attribute, looks straight to base.getattr and misses base.getattro entirely?

If so? Does that qualify as a bug?

Answer 1

References

tp_getattr is deprecated, so tp_getattro would be the better decision to use. But it's not the right method to use at all. Having a look at the Objects/typeobject.c source file of Python, you'll find the definition of the super type, as well as a static function add_methods, which defines the methods of a type. All three together illustrate, why you'll run into problems.

Why not to use tp_getattro to "define" methods

It is usually convenient to set this field to PyObject_GenericGetAttr(), which implements the normal way of looking for object attributes.

So the lookup is only defined for the object and not for the type, but methods should be defined for the type.

How methods are defined

The function add_methods iterates over tp_methods, defines every method with PyDescr_NewMethod (PyCFunction_New for static methods) and adds it to the dictionary of the type object.

How super searches for methods

The super type defines tp_getattro. Now super().func_noargs() will trigger a lookup for func_noargs within the super object. This means, that func_noargs will be looked up in the dictionary of every relevant super type. If nothing is found, __class__ was requested or the second parameter to super was not defined or None, the super object will call PyObject_GenericGetAttr on itself. So your tp_getattro is never called.

Conclusion

If you don't like your original (working) solution, you should put the functions you return in tp_getattro into the dictionary of the type object.