Extending Python with C Error: Segmentation Fault: 11 in Python [MacOS X]

Question

I have written an extension module in C with the name extmodule.c and the code for it is as follows:

#include <Python.h>

//Define a new exception object for our module
static PyObject *extError;

static PyObject* ext_cpu(PyObject* self, PyObject *args)
{
  int pid;
  int sts=0;

  //We expect at least 1 argument to this function
  if(!PyArg_ParseTuple(args, "i", &pid))
  {
    return NULL;
  }


  printf("Hello, from C World! Pid: %i", pid);
  sts=pid;

  return Py_BuildValue("i", sts);
}

static PyMethodDef ext_methods[] = {
  //PythonName, C-FunctionName, argument_presentation, description
  {"cpu", ext_cpu, METH_VARARGS, "Print cpu consumption of a particular process with pid"}
};

PyMODINIT_FUNC
PyInit_ext(void)
{
    PyObject *m;

    m = PyModule_Create(&ext_methods);
    if (m == NULL)
        return NULL;

    extError = PyErr_NewException("spam.error", NULL, NULL);
    Py_INCREF(extError);
    PyModule_AddObject(m, "error", extError);
    return m;
}

After that I have created a setup.py to build and install the extension file in my python program and the code for the setup.py is as follows:

from distutils.core import setup, Extension

module1 = Extension('ext',
    include_dirs = ['/usr/local/include'],
    libraries = ['pthread'],
    sources = ['extmodule.c'])

setup (name = 'ext',
    version = '1.0',
    description = 'This is a C extension for Python program',
    author = 'Somdip Dey',
    url = '',
    ext_modules = [module1])

Now on the command prompt I have built the setup.py using the following commands:

>> python setup.py build

running build running build_ext

building 'ext' extension gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/somdipdey/ anaconda3/include -arch x86_64 -I/Users/somdipdey/anaconda3/include -arch x86_64 -I/usr/local/include -I/Users/somdipdey/anaco nda3/include/python3.6m -c extmodule.c -o build/temp.macosx-10.7-x86_64-3.6/extmodule.o extmodule.c:34:25: warning: incompatible pointer types passing 'PyMethodDef (*)[1]' to parameter of type 'struct PyModuleDef *' [-Wincompatible-pointer-types] m = PyModule_Create(&ext_methods); /Users/somdipdey/anaconda3/include/python3.6m/modsupport.h:158:26: note: expanded from macro 'PyModule_Create' PyModule_Create2(module, PYTHON_API_VERSION) ^~~~~~ /Users/somdipdey/anaconda3/include/python3.6m/modsupport.h:150:60: note: passing argument to parameter here PyAPI_FUNC(PyObject ) PyModule_Create2(struct PyModuleDef, ^ 1 warning generated. gcc -bundle -undefined dynamic_lookup -L/Users/somdipdey/anaconda3/lib -arch x86_64 -L/Users/somdipdey/anaconda3/lib -arch x86 _64 -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/extmodule.o -L/Users/somdipdey/anaconda3/lib -lpthread -o build/lib.macosx- 10.7-x86_64-3.6/ext.cpython-36m-darwin.so

>> python setup.py install

The install command worked properly but the build one gave 1 warning. Now when I am trying to import ext in my python program and use the function ext.cpu(integer_value), the program is giving me the following error:

Segmentation Fault: 11

Any idea what might be causing the issue and how to get rid of it?

Answer 1

The warning is telling you exactly what's wrong: you're passing a PyMethodDef (*)[1] to PyModule_Create, when it expected a PyModuleDef *. Those are completely unrelated types. The segfault you're getting is like the C version of a TypeError.

You need to create a module definition table, and pass that to PyModule_Create.

---

If you fix that, you may or may not have another segfault, or garbage data, or a mysterious segfault on exit, because your method table is missing the empty row at the end. C arrays don't know their size the way Python lists do, so code that uses them either needs to pass around the size in a separate variable, or use some "sentinel" value in the last slot. PyMethodDef uses the latter solution.

---

So:

static PyMethodDef ext_methods[] = {
  //PythonName, C-FunctionName, argument_presentation, description
  {"cpu", ext_cpu, METH_VARARGS, "Print cpu consumption of a particular process with pid"},
  {NULL, NULL, 0, NULL}
};

static struct PyModuleDef ext_module = {
  PyModuleDef_HEAD_INIT,
  "ext",
  "Extension module that does stuff",
  -1,
  ext_methods
};

PyMODINIT_FUNC
PyInit_ext(void)
{
    PyObject *m;

    m = PyModule_Create(&ext_module);
    // the rest is the same as before

With those changes, your module builds without warnings, and:

>>> import ext
>>> ext.cpu(23)
Hello, from C World! Pid: 23
>>> ^D

… everything works fine.

(Well, there might be a memory leak in there, but that's a separate issue…)