Search code examples
pythonpickleboost-python

Pickling boost python functions

For the use with joblib.Parallel, I need to be able to pickle a boost::python function. When I try to do so, I get a

TypeError: can't pickle builtin_function_or_method objects

As far as I understand, the function should be pickled by fully qualified name only. I don't see why this is not possible. Any ideas?

Solution

  • If you want to use your boost method in a joblib.Parallel object, maybe you could use a wrapper around your boost method :

    from joblib import Parallel, delayed
from boost import boost_function

class Wrapper(object):
    def __init__(self, method_name, module_name):
        self.method_name = method_name
        self.module_name = module_name

    def __call__(self, *args, **kwargs):
        method = __import__(self.module_name, globals(), locals(), [self.method_name,])
        return method(*args, **kwargs)

Parallel(n_jobs=1)(delayed(Wrapper("boost_module_name_with_dots", "boost_method_name")(i) for i in range(10))