Search code examples
ruby-on-railsrubymultithreadingconcurrencyconcurrent-ruby

How do I catch an error from a thread and then re-throw that error when all the threads have completed?


I'm using Rails 5. I have this gem for managing threads ...

gem 'concurrent-ruby'

I notice that if one of my threads throws an error, it is just swallowed and I never find out about it. I tried this in a console

pool = Concurrent::FixedThreadPool.new(1)
  # => #<Concurrent::FixedThreadPool:0x007fe3585ab368 @__lock__=#<Thread::Mutex:0x007fe3585ab0c0>, @__condition__=#<Thread::ConditionVariable:0x007fe3585ab098>, @min_length=1, @max_length=1, @idletime=60, @max_queue=0, @fallback_policy=:abort, @auto_terminate=true, @pool=[], @ready=[], @queue=[], @scheduled_task_count=0, @completed_task_count=0, @largest_length=0, @ruby_pid=23932, @gc_interval=30, @next_gc_time=252232.13299, @StopEvent=#<Concurrent::Event:0x007fe3585aaf30 @__lock__=#<Thread::Mutex:0x007fe3585aaeb8>, @__condition__=#<Thread::ConditionVariable:0x007fe3585aae90>, @set=false, @iteration=0>, @StoppedEvent=#<Concurrent::Event:0x007fe3585aadc8 @__lock__=#<Thread::Mutex:0x007fe3585aad78>, @__condition__=#<Thread::ConditionVariable:0x007fe3585aad50>, @set=false, @iteration=0>> 
nums.each do |num|
  pool.post do
    if num == 1
      asdfasdf
    end
  end
end
  # => [1, 2, 3] 
pool.shutdown             # => true 
pool.wait_for_termination # => true 

I was wondering how, if one of the threads from my pool throws an error, I can throw an exception when all the threads have completed, halting my program. If none of the threads throws an error, then I'm fine to continue with whatever was happening.

Above, you'll notice I intentionally cause a condition that should result in an error, but I never find out about it, because I guess the threadpool is swallowing the output of the exceptions.


Solution

  • To answer your question - no actual way as the library explicitly silences exceptions and there is no configuration for it.

    A possible workaround would be to capture exceptions manually:

    error = nil
    pool = Concurrent::FixedThreadPool.new(1)
    numbers.each do |number|
      pool.post do
        begin
          some_dangerous_action(number)
        rescue Exception => e
          error = e
          raise # still let the gem do its thing
        end
      end
    end
    
    pool.shutdown
    pool.wait_for_termination
    
    raise error if error