I am perplexed about the error in my docker-compose config. I am adding redis alongside sidekiq to the existing rails app that uses docker-compose but I am failing hard on attempts to make the containers communicate with each other. I have tried several different options and looked at pretty much every reasonable topic on SO that touches this subject but I keep failing on basically the same thing. Here is my current config (or at least the relevant parts of it) and error:
services:
web:
<<: *common-settings
build:
context: .
dockerfile: docker/development/Dockerfile
target: web
command: dumb-init bash -c "(bundle check || bundle install) && rails s -b 0.0.0.0 -p 3000"
depends_on:
- db
- redis
- elasticsearch
ports:
- '3000:3000'
expose:
- 3000
redis:
image: 'redis:5-alpine'
ports:
- '6379:6379'
volumes:
- 'redis:/data'
sidekiq:
depends_on:
- db
- redis
- web
build: .
command: bundle exec sidekiq
volumes:
- .:/app:cached
- bundle:/bundle
env_file:
- ./.env
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redis:6379/12' }
end
Sidekiq.configure_client do |config|
config.redis = { url: 'redis://redis:6379/12' }
end
Yes this is hardcoded right in the open currently, but I have tried configs that just use ENV variables and the results are exactly the same, no matter where I get the values from the result is the same. Trying out stuff like this right now since it made no difference.
Here is the error when I try to access localhost:3000/sidekiq
Error connecting to Redis on redis:6379 (SocketError)
Application Trace | Framework Trace | Full Trace
redis (4.4.0) lib/redis/client.rb:384:in `rescue in establish_connection'
redis (4.4.0) lib/redis/client.rb:365:in `establish_connection'
redis (4.4.0) lib/redis/client.rb:117:in `block in connect'
redis (4.4.0) lib/redis/client.rb:330:in `with_reconnect'
redis (4.4.0) lib/redis/client.rb:116:in `connect'
redis (4.4.0) lib/redis/client.rb:403:in `ensure_connected'
redis (4.4.0) lib/redis/client.rb:255:in `block in process'
redis (4.4.0) lib/redis/client.rb:342:in `logging'
redis (4.4.0) lib/redis/client.rb:254:in `process'
redis (4.4.0) lib/redis/client.rb:148:in `call'
redis (4.4.0) lib/redis.rb:307:in `block in info'
redis (4.4.0) lib/redis.rb:70:in `block in synchronize'
/usr/local/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize'
redis (4.4.0) lib/redis.rb:70:in `synchronize'
redis (4.4.0) lib/redis.rb:306:in `info'
sidekiq (6.2.2) lib/sidekiq.rb:120:in `block in redis_info'
sidekiq (6.2.2) lib/sidekiq.rb:98:in `block in redis'
connection_pool (2.2.5) lib/connection_pool.rb:63:in `block (2 levels) in with'
connection_pool (2.2.5) lib/connection_pool.rb:62:in `handle_interrupt'
connection_pool (2.2.5) lib/connection_pool.rb:62:in `block in with'
connection_pool (2.2.5) lib/connection_pool.rb:59:in `handle_interrupt'
connection_pool (2.2.5) lib/connection_pool.rb:59:in `with'
sidekiq (6.2.2) lib/sidekiq.rb:95:in `redis'
sidekiq (6.2.2) lib/sidekiq.rb:114:in `redis_info'
sidekiq (6.2.2) lib/sidekiq/web/helpers.rb:177:in `redis_info'
.
.
.
Same error occurs when I try to run a worker from the web app, or go there and use bundle exec sidekiq
Here is what gets logged out when I run docker-compose up
redis_1 | 1:C 12 Oct 2021 11:44:19.103 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 12 Oct 2021 11:44:19.103 # Redis version=5.0.14, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 12 Oct 2021 11:44:19.103 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 12 Oct 2021 11:44:19.105 * Running mode=standalone, port=6379.
redis_1 | 1:M 12 Oct 2021 11:44:19.105 # Server initialized
redis_1 | 1:M 12 Oct 2021 11:44:19.105 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 12 Oct 2021 11:44:19.105 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain
the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 12 Oct 2021 11:44:19.105 * DB loaded from disk: 0.000 seconds
redis_1 | 1:M 12 Oct 2021 11:44:19.106 * Ready to accept connections
sidekiq_1 | 2021-10-12T11:44:26.712Z pid=1 tid=ouccdy1p5 INFO: Booting Sidekiq 6.2.2 with redis options {:url=>"redis://redis:6379/12"}
sidekiq_1 | 2021-10-12T11:44:27.174Z pid=1 tid=ouccdy1p5 INFO: Booted Rails 5.2.6 application in development environment
sidekiq_1 | 2021-10-12T11:44:27.174Z pid=1 tid=ouccdy1p5 INFO: Running in ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux-musl]
sidekiq_1 | 2021-10-12T11:44:27.174Z pid=1 tid=ouccdy1p5 INFO: See LICENSE and the LGPL-3.0 for licensing details.
sidekiq_1 | 2021-10-12T11:44:27.174Z pid=1 tid=ouccdy1p5 INFO: Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org
As for using ENV variables I usually tried it with setting a value like this in the .env file
REDIS_URL=redis://redis:6379/12
When I check with docker ps, redis container is there normally.
If I understand correctly, the URL redis://redis:6379/12 uses redis as the name of the host and that corresponds to the name of the service I specified in the docker-compose.yml and should allow the containers to talk to each other. This file originally has more services like elasticsearch for example and they work just fine.
I have tried quite a few different setups but I feel like I'm banging my head against the fall. What am I missing?
Turns out I missed an important detail about the config
networks:
app:
After adding
networks:
- app
To redis and sidekiq, everything was able to talk to each other nicely.