Why should Py_INCREF(Py_None) be required before returning Py_None in C?

Question

Why should Py_INCREF(Py_None) be required before returning Py_None in C as follows?

Py_INCREF(Py_None);
return Py_None;

If Py_INCREF(Py_None) is omitted, what will happen?

Answer 1

Missing a Py_INCREF will result in an incorrect counting of references for Py_None, which may lead the interpreter to deallocate Py_None. Since Py_None is allocated statically in the Objects/object.c file:

PyObject _Py_NoneStruct = {
  _PyObject_EXTRA_INIT
  1, &PyNone_Type
};

And in Include/object.h there is the define:

#define Py_None (&_Py_NoneStruct)

So what will happen, is that the interpreter will crash with a fatal error:

Fatal Python error: deallocating None

Which is generated by the none_dealloc function in Objects/object.c:

/* ARGUSED */
static void
none_dealloc(PyObject* ignore)
{
    /* This should never get called, but we also don't want to SEGV if
     * we accidentally decref None out of existence.
     */
    Py_FatalError("deallocating None");
}

As stated by that comment, if NoneType didn't have its own deallocating function, you would obtain a Segmentation Fault, since a free call would be done on the stack.

You can test this copying the example in the tutorial, adding a call to Py_DECREF(Py_None) into the Noddy_name function, build the extension and do a loop calling that method.

---

In the general case a reference count of 0 can cause the program to fail in many different ways.

In particular python is free to re-use the memory used by the objects that were deallocated, which means that suddenly every reference to an object can become references to a random object(or to an empty memory location), and you could see things like:

>>> None   #or whatever object that was deallocated
<ARandomObjectYouNeverSawBefore object at ...>

(This actually happened to me sometimes, when writing C extensions. Some objects where turning into read only buffers at random times due to missing calls to Py_INCREF).

In other situations different kind of errors could be raised, or the interpreter could crash or segfault.