Python/C++: can import Armadillo (arma::) but not subroutine arma::arma_rng::randn

Question

QUESTION

When creating a Python extension in C++ that uses Armadillo, I get the errors:

A) In Mac OS Mojave 10.14.4, Python 3.7.5:

Traceback (most recent call last):
  File "./py_program.py", line 5, in <module>
    import cmodule
ImportError: dlopen(/Users/angel/.pyenv/versions/3.7.5/lib/python3.7/site-packages/cmodule.cpython-37m-darwin.so, 2): Symbol not found: __ZTWN4arma23arma_rng_cxx11_instanceE
  Referenced from: /Users/angel/.pyenv/versions/3.7.5/lib/python3.7/site-packages/cmodule.cpython-37m-darwin.so
  Expected in: flat namespace in /Users/angel/.pyenv/versions/3.7.5/lib/python3.7/site-packages/cmodule.cpython-37m-darwin.so

B) In Ubuntu 20, Python 3.8.2:

Traceback (most recent call last):
  File "./py_program.py", line 5, in <module>
    import cmodule
ImportError: /usr/local/lib/python3.8/dist-packages/cmodule.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN4arma23arma_rng_cxx11_instanceE

Both of them due to the use of arma::arma_rng::randn<double>(), see below.

How can I fix it?

DETAILS

I want py_program.py to import the C++ module (extension) defined in cmodule.cpp. Following the documentation https://docs.python.org/3/extending/extending.html , I have the files py_program.py, setup.py and cmodule.cpp. Their contents are:

1. py_program.py

#!/usr/bin/env python3

"""Import and use cmodule."""

import cmodule
cmodule.printvol(3.)

2. setup.py

"""To install the module defined in cmodule.cpp."""
from distutils.core import setup, Extension
setup(name='cmodule', version='1.0',  \
      ext_modules=[
          Extension(
            'cmodule', ['cmodule.cpp'],
            extra_compile_args=['-std=c++11'],
            language='c++')],
      )

3. cmodule.cpp

/* Module to be used in Python.
   All Python stuff follows Sec. 1.8 of 
   https://docs.python.org/3/extending/extending.html */
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <armadillo>


double f()
// Fails at using Armadillo.
// The module works if I delete this function.
{
    double rn_y = arma::arma_rng::randn<double>();
    return rn_y;
}


arma::cx_double g()
// Succeeds at using Armadillo.
{
    arma::cx_double value(0., 1.);
    return value;
}


static PyObject *
cmodule_printvol(PyObject *self, PyObject *args)
// A method of the module.
{
    double voltage;
    if (!PyArg_ParseTuple(args, "d", &voltage))
        return NULL;
    printf("voltage is %f.\n", voltage);
    Py_RETURN_NONE;
}


static PyMethodDef cmodule_methods[] = {
// Declare the modules methods.
    {"printvol", cmodule_printvol, METH_VARARGS, "Print voltage."},
    {NULL, NULL, 0, NULL}   /* sentinel */
};


static struct PyModuleDef cmodule = {
// Create the module.
    PyModuleDef_HEAD_INIT,
    "diff",
    NULL,
    -1,
    cmodule_methods
};

PyMODINIT_FUNC
PyInit_cmodule(void)
// Initialize the module.
{
    return PyModule_Create(&cmodule);
}

I run them as follows:

python setup.py install
python py_program.py

In Ubuntu, the output of python3 setup.py install is:

running install
running build
running build_ext
building 'cmodule' extension
creating build
creating build/temp.linux-x86_64-3.8
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.8 -c cmodule.cpp -o build/temp.linux-x86_64-3.8/cmodule.o -std=c++11
creating build/lib.linux-x86_64-3.8
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/cmodule.o -o build/lib.linux-x86_64-3.8/cmodule.cpython-38-x86_64-linux-gnu.so
running install_lib
copying build/lib.linux-x86_64-3.8/cmodule.cpython-38-x86_64-linux-gnu.so -> /usr/local/lib/python3.8/dist-packages
running install_egg_info
Removing /usr/local/lib/python3.8/dist-packages/cmodule-1.0.egg-info
Writing /usr/local/lib/python3.8/dist-packages/cmodule-1.0.egg-info

What causes the problem is

double rn_y = arma::arma_rng::randn<double>();

Actually, if I delete the function f(), I get no error. Notice that Armadillo is loaded successfully, since g() uses it no problem. What is happening?

Answer 1

In setup.py, the argument libraries=['armadillo'] to Extension() fixes the problem:

"""To install the module defined in cmodule.cpp."""
from distutils.core import setup, Extension
setup(name='cmodule', version='1.0',  \
      ext_modules=[
          Extension(
            'cmodule', ['cmodule.cpp'],
            extra_compile_args=['-std=c++11'],
            libraries=['armadillo'], // this solves the problem
            language='c++')],
      )

Mysteriously, without it, arma:: can be used correctly. But not 'submodules' like arma::arma_rng.

This solution is general: the same problem happens with other libraries. Actually, I reproduced the same (and made it work) with the GNU Scientific Library (libraries=['gsl']).