Search code examples
ruby-on-railsredisdebouncingparallel-testingrspec-sidekiq

How to prevent Rails parallel tests and Sidekiq from calling Redis


I have a Rails application that is using Sidekiq for asynchronous jobs. I setup parallel testing to speed up my testing pipeline. This is working great but I have one major problem: Redis is bottlenecking with Sidekiq jobs trying to enqueue work there.

I keep seeing these errors:

Redis::CannotConnectError:
Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED)

I am running some old versions of gems and also using Sidekiq debounce (Not sure if this is the reason)

sidekiq (5.2.1) sidekiq-debounce (1.1.0) rspec-sidekiq (3.0.3)

I was under the impression that rspec-sidekiq would be able to either stub the calls to Redis by using either: Sidekiq::Testing.inline! or Sidekiq::Testing.fake! to run the jobs inline or push them to an array respectively. However neither of these options are working. What am I missing?

I tried setting up the test environment to fake or inline the jobs like:

require "sidekiq/testing"

Sidekiq::Testing.inline!

OR

require "sidekiq/testing"

Sidekiq::Testing.fake!

Then I even checked in some of the tests to make sure this was being set by:

puts "SIDEKIQ MODE ENABLED: #{Sidekiq::Testing.enabled?}"
puts "SIDEKIQ MODE FAKE: #{Sidekiq::Testing.fake?}"
puts "SIDEKIQ MODE INLINE: #{Sidekiq::Testing.inline?}"
puts "SIDEKIQ MODE DISABLED: #{Sidekiq::Testing.disabled?}"

and it returned what I expected but I still got the Redis connections refused.

I also tried wrapping the individual parts in blocks like:

Sidekiq::Testing.fake! do
 # some code
end

Solution

  • So I think I figured out what was wrong or what to change. We were using Sidekiq along with Sidekiq-debounce I think this was actually intercepting the basic Sidekiq async calls so even though I set them to Sidekiq::Testing.fake! or Sidekiq::Testing.inline!, this did not affect jobs being called using SomeWorker.perform_in()

    I ended disabling the debounce middleware for the test environment in our sidekiq.rb initializer:

    unless Rails.env.test?
      Sidekiq.configure_client do |config|
        config.client_middleware do |chain|
          chain.add Sidekiq::Debounce
        end
      end
    end
    
    if Rails.env.test?
      require 'sidekiq/testing'
      Sidekiq::Testing.fake!
    end

    Ran the pipeline again and no longer saw the Redis connection problems.