Search code examples
pythonpython-2.7built-inpython-c-extensionpython-extensions

Python's __radd__ doesn't work for C-defined types


When creating a python (2.7.5) extension that defines a noddy.Noddy type with __radd__ method, it gets a different behavior from a (otherwise equivalent) python defined-class object with a custom __radd__ (the former does not work, while the latter works). Example:

class PythonClass():
    def __radd__(self, other):
        return 'indeed!'

w = PythonClass()
d = noddy.Noddy()

print(w.__radd__)
print(d.__radd__)

print('the following works:')
print([1] + w)
print('the following does not work:')
print([1] + d)

And the corresponding output:

<bound method PythonClass.__radd__ of <__main__.PythonClass instance at 0xf6e9792c>>
<built-in method __radd__ of noddy.Noddy object at 0xf749d4b8>
the following works:
indeed!
the following does not work:
Traceback (most recent call last):
  File "examples/2.py", line 44, in <module>
    print([1] + d)
TypeError: can only concatenate list (not "noddy.Noddy") to list

The method d.__radd__ is not called, but w.__radd__ is. Any ideas as to why this is so? The behavior of [1] + x where x is a PythonClass instance seems to be in accord with the documentaton, and I would expect noddy.Noddy to work as well. Also, both are types unrelated to list.

Workarounds are welcome. I've already tried patching list.__radd__ with forbiddenfruit, with no success, though I have brought this issue to the author's attention, who happens to be a close friend of mine.

EDIT

...and here is a picture of the C land:

typedef struct {
    PyObject_HEAD
} Noddy;


static PyObject*
Noddy_radd(PyObject* _self, PyObject* args) {
    printf("Noddy_radd!\n");
    return NULL;
}

static PyObject*
Noddy_add(PyObject* _self, PyObject* args) {
  printf("Noddy_add\n");
  return NULL;
}

PyNumberMethods noddy_nums = {
  Noddy_add,         /* binaryfunc nb_add;         /* __add__ */
    0,               /* binaryfunc nb_subtract;    /* __sub__ */
    0,               /* binaryfunc nb_multiply;    /* __mul__ */
    0,               /* binaryfunc nb_divide;      /* __div__ */
    0,               /* binaryfunc nb_remainder;   /* __mod__ */
    0,               /* binaryfunc nb_divmod;      /* __divmod__ */
    0,               /* ternaryfunc nb_power;      /* __pow__ */
    0,               /* unaryfunc nb_negative;     /* __neg__ */
    0,               /* unaryfunc nb_positive;     /* __pos__ */
    0,               /* unaryfunc nb_absolute;     /* __abs__ */
    0,               /* inquiry nb_nonzero;        /* __nonzero__ */
    0,               /* unaryfunc nb_invert;       /* __invert__ */
    0,               /* binaryfunc nb_lshift;      /* __lshift__ */
    0,               /* binaryfunc nb_rshift;      /* __rshift__ */
    0,               /* binaryfunc nb_and;         /* __and__ */
    0,               /* binaryfunc nb_xor;         /* __xor__ */
    0,               /* binaryfunc nb_or;          /* __or__ */
    0,               /* coercion nb_coerce;        /* __coerce__ */
    0,               /* unaryfunc nb_int;          /* __int__ */
    0,               /* unaryfunc nb_long;         /* __long__ */
    0,               /* unaryfunc nb_float;        /* __float__ */
    0,               /* unaryfunc nb_oct;          /* __oct__ */
    0,               /* unaryfunc nb_hex;          /* __hex__ */
};

static PyMethodDef Noddy_methods[] = {
    {"__radd__", (PyCFunction)Noddy_radd, METH_VARARGS,
     "__radd__ function"},
    {NULL}  /* Sentinel */
};


static PyTypeObject NoddyType = {
    PyObject_HEAD_INIT(NULL)
    0,                         /*ob_size*/
    "noddy.Noddy",             /*tp_name*/
    sizeof(Noddy),             /*tp_basicsize*/
    0,                         /*tp_itemsize*/
    0,                         /*tp_dealloc*/
    0,                         /*tp_print*/
    0,                         /*tp_getattr*/
    0,                         /*tp_setattr*/
    0,                         /*tp_compare*/
    0,                         /*tp_repr*/
    &noddy_nums,               /*tp_as_number*/
    0,                         /*tp_as_sequence*/
    0,                         /*tp_as_mapping*/
    0,                         /*tp_hash */
    0,                         /*tp_call*/
    0,                         /*tp_str*/
    0,                         /*tp_getattro*/
    0,                         /*tp_setattro*/
    0,                         /*tp_as_buffer*/
    Py_TPFLAGS_DEFAULT |
      Py_TPFLAGS_HAVE_SEQUENCE_IN | /* tp_flags */
      Py_TPFLAGS_HAVE_ITER,
    "Noddy objects",           /* tp_doc */
    0,                     /* tp_traverse */
    0,                     /* tp_clear */
    0,                     /* tp_richcompare */
    0,                     /* tp_weaklistoffset */
    0,                     /* tp_iter */
    0,                     /* tp_iternext */
    Noddy_methods,             /* tp_methods */
    0,                         /* tp_members */
    0,                         /* tp_getset */
    0,                         /* tp_base */
    0,                         /* tp_dict */
    0,                         /* tp_descr_get */
    0,                         /* tp_descr_set */
    0,                         /* tp_dictoffset */
    0,                         /* tp_init */
    0,                         /* tp_alloc */
    PyType_GenericNew,         /* tp_new */
};

Solution

  • Python's getattr is a tricky guy. The __radd__ method is part of the [in]famous magic methods. They're not stored in the same array as the regular methods (ob_type->tp_methods), it is part of the ob_type->tp_as_number, managed separately by the Number Protocol.

    Forbidden fruit has an issue requesting the ability to monkey patch those methods. This effort is documented here