I'm getting this error from sidekiq / rails / postgresql combination after it processes about 2000 jobs.
PG::ConnectionBad: PQconsumeInput() SSL error: system lib
It's on simple / random SQL queries, sometimes the ActiveRecord table schema queries. Things that work OK for 2000 or so queries suddenly start to fail for some unknown reason.. I get about 50 failures per 10,000 requests, and then at about 50,000 requests sidekiq falls overs and I need to restart it.
I often get before the crash something like..
Celluloid::TimeoutError: linking timeout of 5 seconds exceeded
Has anyone run into this? Hosting is Amazon AWS with RDS for PostgreSQL. It's a recent issue I didn't use to have it and I'm out of ideas so any suggestions would be appreciated.
I worked with Brett to determine the cause. He'd set his Sidekiq concurrency to 100. This is way too high for MRI. He turned it back down to the default of 25 and stability returned. Instead we're going to run 4 processes of 25 threads instead of a single process with 100 threads.