Search code examples
ruby-on-railsrubyredissidekiq

How to set up 'workers' server responsible for running jobs from another servers


Let's say that we have 2 servers. One app server where is http server, app server and application, it would be opened for world. Another server worker is supposed to run the Sidekiq jobs only (has application, Redis and Sidekiq).

How to configure the Sidekiq to enqueue jobs from app server an run them in worker server?


Solution

  • Your app server must have the Sidekiq gem to enqueue the jobs and also have access to your Redis database located in your worker server. I would just use any Redis cloud service that exists out there and let both the app server and worker server have access to it.

    Either way, you basically need to configure both your Sidekiqs in app and worker servers so that they know where to read/write jobs from (what is the Redis database address).

    Check this section of the docs:

    https://github.com/mperham/sidekiq/wiki/Using-Redis

    To sum up, you have two servers that share the same Redis database. One enqueue jobs to it, and the other runs them.

    Another question is that once the workers finish their jobs, what do they do with the results. You have several options in this case: push the results back to a Redis queue, send the results via HTTP to the app server...