Implementing numeric methods always return NotImplemented

Question

I'm writing a new extension type, but I have a problem setting the numeric operations(such as addition/subtraction/multiplication). I have managed to set the some in-place operations, while the normal operations are not called.

For example, I have the function:

static PyObject *
MyType_Mul(PyObject *v, PyObject *w)
{
    PyErr_SetString(PyExc_ValueError, "testing");
    return NULL;
}

And I set it in the numbers methods like this:

static PyNumberMethods my_type_as_number = {
    0,  /* nb_add */
    0,  /* nb_sub */
    (binaryfunc)MyType_Mul,  /* nb_mul */
    ...
    0,  /* nb_in_place_add */
    0,  /* nb_in_place_sub */
    (binaryfunc)MyType_Mul,  /* nb_in_place_mul */
    ...
};

Now when I try to use my type I get this kind of behaviour:

>>> from mytype import MyType
>>> a = MyType()
>>> a * 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for *: 'mytype.MyType' and 'int'
>>> 2 * a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for *: 'int' and 'mytype.MyType'

But if I use the in-place operator:

>>> a *= 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: testing

If I use dir() on the object I can see the __mul__ and __rmul__ methods(which means that python sees them), but it seems like they are not called at all. Using a.__mul__(2) returns NotImplemented.

Also:

>>> a.__mul__
<method-wrapper '__mul__' of mytype.MyType object at 0x7fc2ecc50468>
>>> a.__imul__
<method-wrapper '__imul__' of mytype.MyType object at 0x7fc2ecc50468>

so, as you can see, they are exactly the same thing.

What's going on? Why the same exact function works for the in-place operator but not the "normal" operator? I've also thought that I might used the wrong slot, but I double checked and it is correct, and also setting it to nb_add, nb_sub etc. does not work.

Answer 1

Thanks to nneonneo's comments I understood what was wrong. Basically I forgot to set the Py_TPFLAGS_CHECKTYPES flag.

There are some clues about this absence in the description I gave:

1. The methods are the same object, but it acts differently for in-place operations
2. In the comments I also said that, for example, doing a*a would yield the correct result, while doing a*different-type does not.

This clearly means that the interpreter, when doing non-in-place operations, is checking the types of the arguments and calls my function if the type is MyType, otherwise returns NotImplemented.

Searching a bit in the documentation it's quite easy to see that this is the default behaviour for numeric methods.

If the argument type is not of the same class, then it is assumed that the operation is not implemented.

To allow different types to be "operated" together you have to set the Py_TPFLAGS_CHECKTYPES flag in the MyType:

static PyTypeObject MyType = {
    PyObject_HEAD_INIT(&PyType_Type)
    0,                         /*ob_size*/
    "mytype.MyType",           /*tp_name*/
    sizeof(MyTypeObject),      /*tp_basicsize*/
    0,                         /*tp_itemsize*/
    ...
    0,                         /*tp_repr*/
    &mytype_as_number,         /*tp_as_number*/
    0,                         /*tp_as_sequence*/
    ...
    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES,/*tp_flags*/
    ...
};

With this flag set the interpreter wont check the types and so you have to deal with them manually.

The in-place operators, instead, always allow different types. Why is that, I don't know.