Cython: using distutils to compile C++

Question

I am trying to wrap some C++ neural network code, so that I have:

numpy -> double* -> armadillo

Anyhow, I am getting the typical:

ImportError: dynamic module does not define init function

It is obviously something wrong in my distutils configuration. Here is a MEW of my project in case someone can finger down the bug:

setup.py

#!/usr/bin/env python build_ext --inplace
# -*- coding: utf-8 -*-

# to run:
# python setup.py build_ext --inplace

from distutils.core import setup, Extension
from Cython.Build import cythonize

setup(
    name='My Test',
    ext_modules = cythonize(Extension(
        "test",
        sources=["nnet_wrap.pyx", "nnet.cpp"],
        libraries = ['armadillo'],
        language="c++",
)))

nnet_wrap.pyx

import numpy as np
cimport numpy as np

cdef extern from "nnet.h":
    void nnet_fit(int D, int N, double* X, double* Y)


class WrapNN:
    def nnet_fit(self, np.ndarray[np.double_t, ndim=2] X):
        X = np.ascontiguousarray(X)
        cdef np.ndarray[np.double_t, ndim=1] y = np.zeros((len(X),), dtype=np.double)
        nnet_fit(X.shape[1], X.shape[0], &X[0,0], &y[0])
        return y

nnet.h

void nnet_fit(int D, int N, double* X, double* Y);

nnet.cpp

#include <armadillo>
using namespace arma;

void nnet_fit(int D, int N, double* X, double* Y)
{
    mat _X(X, N, D, false);
    // do work
    vec _Y;  // results
    for(int i = 0; i < N; i++)
        Y[i] = _Y[i];
}

I know it is a little hacky. Anyhow, it compiles fine and generates a test.so file, but then it fails to import:

$ python setup.py build_ext --inplace
$ python
>>> import test
ImportError: dynamic module does not define init function (inittest)

EDIT: some more information as requested:

$ readelf -Ws test.so | grep init
   118: 0000000000003168     0 FUNC    GLOBAL DEFAULT    9 _init
   123: 000000000000ab90   160 FUNC    WEAK   DEFAULT   11 _ZN4arma3MatIdE9init_coldEv
   126: 0000000000009ae0  4211 FUNC    GLOBAL DEFAULT   11 initnnet_wrap
    42: 00000000000036e0    79 FUNC    LOCAL  DEFAULT   11 _ZL30__Pyx_CyFunction_init_defaultsP22__pyx_CyFunctionObject
   202: 000000000020f3e0     1 OBJECT  LOCAL  DEFAULT   24 _ZStL8__ioinit
   211: 000000000020dce0     0 OBJECT  LOCAL  DEFAULT   17 __frame_dummy_init_array_entry
   306: 0000000000009ae0  4211 FUNC    GLOBAL DEFAULT   11 initnnet_wrap
   330: 000000000000ab90   160 FUNC    WEAK   DEFAULT   11 _ZN4arma3MatIdE9init_coldEv
   344: 0000000000003168     0 FUNC    GLOBAL DEFAULT    9 _init

By the way, my Cython version is the one from Ubuntu 14.04 which could be slightly outdated. I know some syntax has been changed from distutils. Tell me if you think that could be a problem.

$ dpkg -s cython | grep Version
Version: 0.20.1+git90-g0e6e38e-1ubuntu2

Answer 1

The module that is being built is called nnet_wrap, but the output file is called test.so. This is confirmed by the fact that your test.so contains a symbol named initnnet_wrap (Python instead is incorrectly looking for inittest).

In your extension definition, replace 'test' with 'nnet_wrap', so that Python can find the right symbol.

ext_modules = cythonize(Extension(
    "test",
#    ^^^^
    sources=["nnet_wrap.pyx", "nnet.cpp"],
#             ^^^^^^^^^
    libraries = ['armadillo'],
    language="c++",
))