Search code examples
pythondistutilspython-c-apicextension

Python CExtension gives ModuleNotFoundError


I am using Python 3 with Anaconda Spyder on CentOS 7. I am trying to get the Python CExtension working by following this video. https://www.youtube.com/watch?v=a65JdvOaygM

I have all of my files in /home/peter/pythonCExtensions which is in the PYTHONPATH

I have a file myModule.c that has the following contents.

#include <stdio.h>
#include <stdlib.h>
#include "/home/peter/anaconda3/include/python3.7m/Python.h"

int Cfib(int n)
{
    if (n<2){
        return n;
    } else {
        return Cfib(n-1)+Cfib(n-2);
    }
}

static PyObject* fib(PyObject* self, PyObject* args)
{
int n;

if (!PyArg_ParseTuple(args, "i", &n))
    return NULL;

    return Py_BuildValue("i", Cfib(n));
}

static PyObject* version(PyObject* self){
    return Py_BuildValue("s", "Version 1.0");
}

static PyMethodDef myMethods[] = {
    {"fib", fib, METH_VARARGS, "Calculates the Fibronacci numbers"},
    {"version", (PyCFunction)version, METH_NOARGS, "Returns the version"},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef myModule = {
    PyModuleDef_HEAD_INIT,
    "myModule",
    "Fibronacci Module",
    -1,
    myMethods
};

PyMODINIT_FUNC PyInit_myModule(void){
    return PyModule_Create(&myModule);
}

I made another file, setup.py, that has the following contents.

from distutils.core import setup, Extension

module = Extension("MyModule", sources = ["myModule.c"])

setup(name="PackageName",
        version="1.0",
        description="This is a package for myModule",
        ext_modules = [module])

I ran

python3 setup.py install

and got

running install
running build
running build_ext
running install_lib
copying build/lib.linux-x86_64-3.7/MyModule.cpython-37m-x86_64-linux-gnu.so -> /home/peter/anaconda3/lib/python3.7/site-packages
running install_egg_info
Writing /home/peter/anaconda3/lib/python3.7/site-packages/PackageName-1.0-py3.7.egg-info

I then copied MyModule.cpython-37m-x86_64-linux-gnu.so and myModule.o into /home/peter/pythonCExtensions.

I then opened Spyder and created a file, CInterface.py, that contains only

import myModule

However, when I run this file (F5) I get

Traceback (most recent call last):

  File "<ipython-input-1-c29fad851da0>", line 1, in <module>
    runfile('/home/peter/Simple3dShapes/CInterface.py',     wdir='/home/peter/Simple3dShapes')

  File "/home/peter/anaconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 827, in runfile
    execfile(filename, namespace)

  File "/home/peter/anaconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/peter/Simple3dShapes/CInterface.py", line 19, in <module>
import myModule

ModuleNotFoundError: No module named 'myModule'

sudo yum install tree

resulted in

Install 1 Package

Total download size: 46 k
Installed size: 87 k
Is this ok [y/d/N]: y
Downloading packages:
tree-1.6.0-10.el7.x86_64.rpm                                                                                                                                                                    |  46 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : tree-1.6.0-10.el7.x86_64                                                                                                                                                                            1/1 
  Verifying  : tree-1.6.0-10.el7.x86_64                                                                                                                                                                        1/1 

Installed:
  tree.x86_64 0:1.6.0-10.el7                                                                                                                                                                                       

Complete!

Solution

  • Listing [Python.Docs]: Building C and C++ Extensions.

    At its core, this is a typo (upper vs. lower case), combined with the fact that on Nix, file names are case sensitive (most likely, it couldn't be reproduced on Win).

    So, your module is called myModule (function PyInit_myModule), but it resides in a file called MyModule.cpython-37m-x86_64-linux-gnu.so, which is not OK because the 2 names must match.
    Correct that by specifying the proper extension name in setup.py:

    module = Extension("myModule", sources = ["myModule.c"])
    

    As a side remark, don't name the module myModule (in general don't name stuff myStuff - personally, I truly hate it :) ), such a name states that the purpose is unclear. You could use e.g. fibonacci_module as a name.