Search code examples
cpythonpypy

PyPy cpyext: any documentation? how to use? PyThreadState_Get error?

I have read (here) that PyPy has support for CPython extension modules via cpyext.

I haven't found any cpyext documentation. Is there any?

How do I use it?

From the source code (e.g. here), I figured out that to load my leveldb.so module, I probably have to do this:

import cpyext
cpyext.load_module("leveldb.so","leveldb")

However, this crashes with this error:

Fatal Python error: PyThreadState_Get: no current thread

I noticed in the backtrace that it calls the functions from my CPython, not from PyPy:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib          0x00007fff8b3e4d46 __kill + 10
1   libsystem_c.dylib               0x00007fff927a9df0 abort + 177
2   org.python.python               0x0000000104692eaa Py_FatalError + 49
3   org.python.python               0x0000000104691370 PyThreadState_Get + 28
4   org.python.python               0x000000010468cf16 Py_InitModule4_64 + 58
5   leveldb.so                      0x00000001027e0881 initleveldb + 49 (leveldb_ext.cc:59)
6   pypy                            0x0000000100f59bb3 PyLong_CheckExact + 55379
7   pypy                            0x0000000100f6e7c7 PyLong_CheckExact + 140391
....
Solution

  • I figured it out.

    It is necessary to recompile the module. I must use the header files from PyPy. For linking, I must not link against libpython. I must just tell the linker to ignore about unresolved symbols.

    This where my commands to build py-leveldb on MacOSX:

    cc -I /usr/local/Cellar/pypy/1.9/include -g -c leveldb_ext.cc leveldb_object.cc
libtool -dynamic -o leveldb.so leveldb_ext.o leveldb_object.o -lleveldb -lsnappy -lc -lstdc++ -undefined dynamic_lookup

    That worked. I just did cpyext.load_module("leveldb.so","leveldb").

    I also found some more notes about CPyExt in their wiki here.