How to access Python dunder methods in C using CFFI?

Question

I was trying to follow this example to attempt to create a simple hash table in C that I could use in Python (using CFFI). Following are the corresponding files and relevant code.

hash_table.h:

typedef struct {
    size_t size;
    ...
    int cmp_function(const PyObject*, const PyObject*);
} hash_table_t;

build_hash_table.py:

from cffi import cffi

HEADER_FILE_NAME = "hash_table.h"
SOURCE_FILE_NAME = "hash_table.c"

ffi = FFI()

header = open(HEADER_FILE_NAME, "rt").read()
source = open(SOURCE_FILE_NAME, "rt").read()

ffi.set_source("hashtable.hash_table", header + source)
ffi.cdef(header)

if __name__ = "__main__":
    ffi.compile()

Now, everything so far works. However, I am going to want to insert whatever data types in the hash table. How do I get access to whatever object I am receiving within the C code? For example, say I had this class in Python:

class Person():

    def __init__(self, name, id):
        self.name = name
        self.id = id

    def __eq__(self, other):
        return self.id == other.id

When I am searching for Person objects in the hash table, how do I access the __eq__ method? As you can see above, I have a generic compare function declared as int cmp_function(const void*, const void*); in the hash table. The goal would be that every time I search for an object, I can know the __eq__ method definition of that object - in the C side.

Hope I made my question clear, thank you in advance!

Answer 1

See https://cffi.readthedocs.io/en/latest/using.html#extern-python-and-void-arguments. Avoid using PyObject * with CFFI. The correct answer is to keep int cmp_function(const void*, const void*);. I will assume that you already use ffi.new_handle() to convert the objects to the void * equivalent that you can send to C (in the add_to_hash_table() method, etc.) and ffi.from_handle() to read them back (in the get_from_hash_table() method, etc.).

To implement the comparison function, you declare a function in Python with extern "Python", as in the above document. Then you use that function for the cmp_function pointer in your struct. Inside that Python function, you receive the two arguments as two void *. You first convert them back to the original Python objects with ffi.from_handle(), and then just use regular Python---in your case, probably just if x == y: return 1; else: return 0 or similar.

Be careful about keeping the Python objects alive after calling ffi.new_handle(). They are not kept alive just by the fact that their void * representation happens to be stored inside some C structures.

Note that CFFI is not quite well suited to implement pure data structures, because all these conversions and multiple storages that increase the overhead when compared with the direct CPython-C-API approach.