python -X showrefcount reporting negative reference counts for extension

Question

When I run cpython with the -X showrefcount flag on an extension I'm writing, it reports a negative reference count (e.g. [-5538 refs, 13503 blocks]) when I return None from a function (using the Py-RETURN_NONE macro).

Known facts:

• The exact count varies between runs, but remains within the same order of magnitude.
• Whatever is happening, it seems to happen slowly; I need to call the extension function approximately 50,000 times before the reference count goes negative.
• If we replace Py_RETURN_NONE; with Py_INCREF(Py_None); return Py_None;, it changes nothing. Indeed, we can seemingly add an arbitrary number of Py_INCREF(Py_None)s without affecting the reference count at all.
• If we replace Py_RETURN_NONE; with return Py_None; and don't increment the reference count, it segfaults (as expected).
• If we replace the None return with another value, e.g. PyLong_FromLong(0);, the problem vanishes.

What is the cause of this? Related question: why is the reference count not zero after running an empty script?

Minimal Example:

build command used for cpython debug build

$ ./configure --with-pydebug --with-valgrind && make

extension.c

#define PY_SSIZE_T_CLEAN
#include <Python.h>

static PyObject *f(void) {
    int manifest_bug = 1;
    if (manifest_bug) {
        Py_RETURN_NONE;
    }
    else {
        return PyLong_FromLong(0);
    }
}

static PyMethodDef functions[] = {
    {"f", (PyCFunction)f, METH_NOARGS, "" },
    {NULL, NULL, 0, NULL},
};

static struct PyModuleDef module = {
    PyModuleDef_HEAD_INIT,
    .m_name = "foo",
    .m_size = -1,
    .m_methods = functions,
};

PyMODINIT_FUNC PyInit_foo(void) {
    return PyModule_Create(&module);
}

setup.py

from setuptools import setup, Extension

name="foo"
def main() -> None:
    setup(
        name=name,
        version="0.0.0",
        ext_modules=[ Extension(name, ["extension.c"]) ],
    )

if __name__ == "__main__":
    main()

test.py

import foo
# With fewer than roughly this many iterations, the reference count
# generally remains positive. With more iterations, it becomes more negative
for i in range(50000):
    foo.f()

Answer 1

The problem was due to the extension having been built using an older version of python, and run using a debug build compiled from the latest version of the source. Extensions not compiled using the stable ABI (and declared as doing so) are not binary compatible across python versions.

[Credit to ead's comment for asking the question that led directly to this solution.]