My goal is to configure SWIG with Python
numpy if a specific numpy version is available andnumpy when a specific numpy version is missing. When the API function that supports numpy is called by the user, an error message should be issued.I have tried to integrate numpy with SWIG in the following way:
%include "numpy.i"
%init %{
import_array();
%}
%apply (double* IN_ARRAY1, int DIM1) {(const double* mymatrix, int n)};
But when I import my SWIG generated Python module I get the following error:
numpy.core.multiarray failed to import
As far as I know this issue can be solved by updating the numpy package to the latest version (currently I have 1.12.1 installed).
But I cannot expect everyone using my API to do the same (install/update numpy), especially when someone does not want to use that one API function.
So this is definitely possible to do. Assuming the goal is to build a shared object/DLL once and run it on multiple systems, some of which will have an appropriate numpy version installed. Our goal is to cause the numpy specific functionality to error gracefully when at runtime there is no numpy support available.
To achieve this we've got two issues to solve:
We can do this in a number of different ways. Firstly though I defined some functionality for testing, with run.py as:
import test
test.printit() # Must always work
print(repr(test.range(100))) # Must either fail gracefully or succeed if numpy usable
These functions are implemented in my testing as:
%inline %{
static void printit(void) {
fprintf(stderr, "Hello world\n");
}
static void range(int *rangevec, int n)
{
int i;
for (i=0; i< n; i++)
rangevec[i] = i;
}
%}
where range comes straight from some numpy documentation
The first idea that springs to mind here is to wrap our call to import_array in our module init with some C API code that tests for an exception and clears it but sets a flag if this occurs.
We can solve problem #1 as suggested using the exception handling of Python's C API:
%{
#define SWIG_FILE_WITH_INIT
static int have_numpy;
%}
%include "numpy.i"
%init %{
// Try and run import_array() - at build time we need it to be defined
_import_array();
// But at runtime we check if it worked or not:
if (PyErr_Occurred()) {
// Exception occurred during import of numpy
have_numpy = 0;
PyErr_Clear();
}
else {
have_numpy = 1;
}
fprintf(stderr, "Runtime numpy check: %d\n", have_numpy);
%}
Note that we had to call _import_array instead of import_array here as import_array was also catching the exception and printing lots of unhelpfully verbose stuff when the import failed in some of my testing.
This is all well and good, but without a solution to issue 2 we won't be failing gracefully when numpy imports fail:
Runtime numpy check: 0
Hello world
Segmentation fault
Opps! So we need a way to hook into the calling of range, but not printit and then raise an exception nicely. For this first case I looked at a few things. Initially I tried to use %exception, since that has nice function matching syntax, however the hook you get for this is too late into the call, we've already used the numpy typemaps and explode before we can prevent it using that.
Instead I used -debug-tmsearch argument to SWIG to pick a typemap that met two properties: firstly numpy itself doesn't already use it (this keeps things simpler) and secondly it occurs early on during the call. The default typemap makes a good candidate for this, so we can use that (with a macro to avoid repetition) nicely in a complete SWIG interface that meets our initial goals:
%module test
%{
#define SWIG_FILE_WITH_INIT
static int have_numpy;
%}
%include "numpy.i"
%init %{
// Try and run import_array() - at build time we need it to be defined
_import_array();
// But at runtime we check if it worked or not:
if (PyErr_Occurred()) {
// Exception occurred during import of numpy
have_numpy = 0;
PyErr_Clear();
}
else {
have_numpy = 1;
}
fprintf(stderr, "Runtime numpy check: %d\n", have_numpy);
%}
// Handle this gracefully
%define %requires_numpy(fn)
%typemap(default) fn {
if (!have_numpy) {
PyErr_SetString(PyExc_NotImplementedError, "Unimplemented without numpy runtime");
SWIG_fail;
}
}
%enddef
// For testing below:
// This must be before the %apply or typemap use
%requires_numpy((int *ARGOUT_ARRAY1, int DIM1));
%apply (int* ARGOUT_ARRAY1, int DIM1) {(int* rangevec, int n)}
%inline %{
static void printit(void) {
fprintf(stderr, "Hello world\n");
}
static void range(int *rangevec, int n)
{
int i;
for (i=0; i< n; i++)
rangevec[i] = i;
}
%}
Without numpy we get this from run.py now:
Runtime numpy check: 0
Hello world
Traceback (most recent call last):
File "run.py", line 5, in <module>
print(repr(test.range(100)))
NotImplementedError: Unimplemented without numpy runtime
With numpy we get:
Hello world
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67,
68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84,
85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99],
dtype=int32)
We can implement our exception throwing in Python also, using %pythonprepend if we prefer. This is the same as before, except we use Python to raise the exception and have exposed have_numpy to the Python module for it to work with:
%module test
%{
#define SWIG_FILE_WITH_INIT
static int have_numpy;
%}
const int have_numpy;
%include "numpy.i"
%init %{
// Try and run import_array() - at build time we need it to be defined
_import_array();
// But at runtime we check if it worked or not:
if (PyErr_Occurred()) {
// Exception occurred during import of numpy
have_numpy = 0;
PyErr_Clear();
}
else {
have_numpy = 1;
}
fprintf(stderr, "Runtime numpy check: %d\n", have_numpy);
%}
// Handle this gracefully
%define %requires_numpy(fn)
%pythonprepend fn %{
if not have_numpy: raise NotImplementedError("This requires numpy")
%}
%enddef
// For testing below:
%requires_numpy(range);
%apply (int* ARGOUT_ARRAY1, int DIM1) {(int* rangevec, int n)}
%inline %{
static void printit(void) {
fprintf(stderr, "Hello world\n");
}
static void range(int *rangevec, int n)
{
int i;
for (i=0; i< n; i++)
rangevec[i] = i;
}
%}
I slightly prefer this given it means we can now name the functions that need numpy instead of interfering with their typemaps.
Finally another alternative would be to split our module into two separate modules. By putting the numpy specific functions into another module and attempting to import that we can achieve the same behaviour still. We can use %import to still share some types and code between the two modules. I've not completed an example of this as it's (possibly?) more complicated than you were looking for, but an outline would be something like:
%module test
%inline %{
static void printit(void) {
fprintf(stderr, "Hello world\n");
}
%}
%pythoncode %{
try:
from test_numpy import *
except:
pass # This is fine
%}
and
%module test_numpy
%import "test.i"
%{
#define SWIG_FILE_WITH_INIT
%}
%include "numpy.i"
%init %{
// Try and run import_array() - at build time we need it to be defined
_import_array();
%}
%apply (int* ARGOUT_ARRAY1, int DIM1) {(int* rangevec, int n)}
%inline %{
static void range(int *rangevec, int n)
{
int i;
for (i=0; i< n; i++)
rangevec[i] = i;
}
%}
Something like this should work as an alternative to the first two options, but is currently untested.