Search code examples
rubymultithreadingconcurrencycelluloid

How can I parallize the execution of my plugins with Celluloid?

My question should help me to get on the right way.

I'm developing a ruby application with the concurrent framework celluloid. Here how it looks like:

activity

I have some plugins. I want to run them concurrently and wait until the last one has finished. I've an abstract class, called PluginFrame, which is inherited by all the plugin and which also provides a run method.

  1. My idea was to make a SupervisionGroup, but is that the right idea?
  2. How can I run a SupervisionGroup and wait until all group members have finished?
  3. It's a good idea to make a separate PluginPool class, to manage the pool of plugins?
  4. It's also interesting for me to limit the pool size, so only two or three plugins run at the same time.How can I achieve this?
Solution

  • You just need a supervisor if your plugins might crash and you want celluloid to restart them.

    What you want to do could be as simple as using pools and futures.

    To have a shared pool for your plugins you need a new actor for that.

    class PluginRunner
  include Celluloid

  def run(plugin)
    plugin.run
  end
end

plugin_runner = PluginRunner.pool(size: 4)

plugins = [LastFmPlugin, TwitterPlugin]
results = plugins.map {|p| plugin_runner.future.run(p) }.map(&:value)
persist_results(results)