AttributeError: undefined symbol when importing own c compiled function in python

I'm trying to domesticate the numba cfunc compiler ;)

This here is my base_model.py file (the source of my function).

import numpy
import numba
import numba.pycc

cc = numba.pycc.CC('base_model')
cc.verbose = True
@cc.export('cumulativeMin','float64[:](float64[:])')  
def cumulativeMin(A):
    r = numpy.empty(len(A))
    t = numpy.inf
    for i in range(len(A)):
        t = numpy.minimum(t, A[i])
        r[i] = t
    return r


if __name__ == "__main__":
    cc.compile()

Then I do (in the terminal, I run ubuntu):

$ python3 base_module.py 
base_model.py:3: NumbaPendingDeprecationWarning: The 'pycc' module is pending deprecation. Replacement technology is being developed.

Pending Deprecation in Numba 0.57.0. For more information please see: https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-the-numba-pycc-module
  import numba.pycc

then I do (in python):

import numpy
import ctypes


mylib = ctypes.cdll.LoadLibrary('./base_model.cpython-310-x86_64-linux-gnu.so')
array = numpy.random.uniform(-1,0,1000)
mylib.cumulativeMin(array)

then I get this error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], line 7
      5 mylib = ctypes.cdll.LoadLibrary('./base_model.cpython-310-x86_64-linux-gnu.so')
      6 array = numpy.random.uniform(-1,0,1000)
----> 7 mylib.cumulativeMin(array)

File /usr/lib/python3.10/ctypes/__init__.py:387, in CDLL.__getattr__(self, name)
    385 if name.startswith('__') and name.endswith('__'):
    386     raise AttributeError(name)
--> 387 func = self.__getitem__(name)
    388 setattr(self, name, func)
    389 return func

File /usr/lib/python3.10/ctypes/__init__.py:392, in CDLL.__getitem__(self, name_or_ordinal)
    391 def __getitem__(self, name_or_ordinal):
--> 392     func = self._FuncPtr((name_or_ordinal, self))
    393     if not isinstance(name_or_ordinal, int):
    394         func.__name__ = name_or_ordinal

AttributeError: ./base_model.cpython-310-x86_64-linux-gnu.so: undefined symbol: cumulativeMin

#Edit:

I would like this function (cummulativeMin) to be compiled ahead of time (https://numba.pydata.org/numba-doc/dev/user/pycc.html).

Solution

That is not how you are supposed to import numba/pycc generated code.

ctypes is for using external library, not compiled for python. It is simpler to build, sure (especially for C coders. You just code C like usually, without caring about python, and let ctypes — with some help from explicit wrapping — handle it). You can even use any already existing .so not written from that purpose. But it is also less sure. You can segfault easily just by giving wrong argument. And you have to manage C world memory by yourself. And can't mess directly with python data.

Here this is cpython extension. So "pure python" functions written in C, if I may use this oxymoron. Functions that use python memory management (garbage collector) to create python data. Harder to write, since you have to read and write python internal data format (with the provided python API, sure). But of course, a better choice when code is generated anyway.

Whatever the criteria, anyway, point is, it is a python module, that is supposed to be imported in python code, that has access to cpython api.

Long story short:

import base_model
base_model.cumulativeMin(...)

is the way you are supposed to use it.

Edit

Since my answer was, I am pretty sure, not understood, I explicit it more (even if that is mostly by repeating the question)

What I did is

Create MySource.py file

import numpy
import numba
import numba.pycc

cc = numba.pycc.CC('base_model')
cc.verbose = True
@cc.export('cumulativeMin','float64[:](float64[:])')  
def cumulativeMin(A):
    r = numpy.empty(len(A))
    t = numpy.inf
    for i in range(len(A)):
        t = numpy.minimum(t, A[i])
        r[i] = t
    return r


if __name__ == "__main__":
    cc.compile()

Then I do (in the terminal, I run ubuntu):

$ python3 MySource.py 
MySource.py:3: NumbaPendingDeprecationWarning: The 'pycc' module is pending deprecation. Replacement technology is being developed.

Pending Deprecation in Numba 0.57.0. For more information please see: https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-the-numba-pycc-module
  import numba.pycc

then I do (in python):

import numpy
import base_model
array = numpy.random.uniform(-1,0,1000)
base_model.cumulativeMin(array)

And I get no error, but the result

array([-0.00492896, -0.80422352, -0.80422352, -0.80422352, -0.80422352,
       -0.80422352, -0.8844663 , -0.8844663 , -0.8844663 , -0.8844663 ,
       -0.8844663 , -0.90776422, -0.90776422, -0.90776422, -0.90776422,
...
       -0.99972223, -0.99972223, -0.99972223, -0.99972223, -0.99972223,
       -0.99972223, -0.99972223, -0.99972223, -0.99972223, -0.99972223])

Note that, outside what is my main answer (don't load it with ctypes, load it with import), I've changed only one thing in my answer: I've changed the name of the source file, from base_model.py to mySource.py. To make one point clear: what import base_model imports here is the .so not a .py file. It IS the compiled file that is loaded.