Search code examples
pythonpython-3.xcythonraycythonize

Cythonize ray actor class


I'm using Ray library and then I want to Cythonize my package. While there is a reference of how to adapt regular remote function

some_cython_func = ray.remote(some_cython_module.some_cython_func)

it's not clear how to deal with Ray Actor (decorator on a class level)

@ray.remote
class MyService:
    def __init__(self):
        pass
    def run(self):
        ...
    def helper(self):
        ...

After being cythonized this code produces such an error

  File "/Users/user/anaconda3/envs/ray-test/lib/python3.7/site-packages/ray/actor.py", line 538, in _remote
    meta.method_meta.methods.keys())
  File "/Users/user/anaconda3/envs/ray-test/lib/python3.7/site-packages/ray/function_manager.py", line 358, in export_actor_class
    "class": pickle.dumps(Class),
  File "/Users/user/anaconda3/envs/ray-test/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 70, in dumps
    cp.dump(obj)
  File "/Users/user/anaconda3/envs/ray-test/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 656, in dump
    return Pickler.dump(self, obj)
_pickle.PicklingError: Can't pickle <cyfunction MyService.__init__ at 0x7fc230367c80>: it's not the same object as app.my_service.__init__

How to rewrite/adapt existing working ray code to work with Cython?

Python 3.7.9 / ray==0.8.6


Solution

  • Yes, seems like a simple solution as

    MyService = ray.remote(_MyService)
    

    works without issue.

    Ref to the Ray source code