Search code examples
mysqlmultithreadingruby-on-rails-4database-connectionsidekiq

How to prevent "Mysql2::Error: This connection is in use by" with Sidekiq


After running Sidekiq for a couple of hours, I see a bunch of jobs fail with Mysql2::Error: This connection is in use by: #<Celluloid::Thread:0x0000000d1b56e0 sleep>. Seems the Sidekiq threads are somehow conflicting over the MySQL connection pool.

concurrency is set to the default 25 in sidekiq.yml and the pool is 28 in database.yml. There are no long-lived queries and the exceptions happen in standard finder calls, nothing fancy.

How can I prevent this error to ensure jobs run smoothly?


Solution

  • Finally fixed this by isolating all the workers. It came down to Typhoeus library apparently not being thread-safe. Replaced with Net/HTTP and it works again.

    Some more details at https://github.com/mperham/sidekiq/issues/1400#issuecomment-45838886