Search code examples
python-extensions

Why does this Py_DECREF cause a segfault?


I'm chasing an annoying segfault bug in a Python extension. Drilling down to the core I first created a standalone C version of the extension, and while trying to further reduce the problem, I've ended up with this. It is the complete program:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

int main(void) {
    PyObject *num;
    PyObject *args;
    num = PyLong_FromLong(0);
    if (!num) return -1;
    args = PyTuple_Pack(1, num);
    if (!args) return -1;
    Py_DECREF(args); /* <-- segfault */
    return 0;
}

If I leave out the Py_DECREF, I don't get an error. According to the docs, PyTuple_Pack returns a new reference, which I now "own". Shouldn't I be allowed to DECREF it?

Running in valgrind, the relevant error message is this:

==7160== Memcheck, a memory error detector
==7160== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==7160== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==7160== Command: ./pyctest
==7160==
==7160== Invalid read of size 4
==7160==    at 0x4F2B13E: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==7160==    by 0x40078B: main (pyctest.c:11)
==7160==  Address 0xa0 is not stack'd, malloc'd or (recently) free'd
==7160==
==7160==
==7160== Process terminating with default action of signal 11 (SIGSEGV)
==7160==  Access not within mapped region at address 0xA0
==7160==    at 0x4F2B13E: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==7160==    by 0x40078B: main (pyctest.c:11)

EDIT

Some build details. This is on a RHEL7 Linux system .

$ python3-config --cflags
-I/usr/include/python3.6m -I/usr/include/python3.6m  -Wno-unused-result -Wsign-compare -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv   -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv
$ python3-config --ldflags
 -L/usr/lib64 -lpython3.6m -lpthread -ldl  -lutil -lm  -Xlinker -export-dynamic
$ cat Makefile
LDFLAGS=-L/usr/lib64 -lpython3.6m -lpthread -ldl  -lutil -lm  -Xlinker -export-dynamic
CFLAGS=-I/usr/include/python3.6m -I/usr/include/python3.6m  -Wno-unused-result -Wsign-compare -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv   -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv


pyctest: pyctest.c
        gcc $(CFLAGS) pyctest.c -o pyctest $(LDFLAGS)
$

Solution

  • The segfault probably happened due to undefined behavior caused by mix-up between Pyhton debugging and production environments.