Ruby on Rails constantly updating a variable

Question

Currently I am building an application that allows users to place bids on products and admins to approve them. The 'transactions' themselves take place outside of the scope of the application. Currently, users see the price of an asset on the transaction/new page and submit a bid by submitting the form. Admins click a button to approve the bid.

class TransactionsController < ApplicationController
before_action :get_price

  def new
    @price = get_price
    @tansaction = Transaction.new
  end

  ###

  def get_price
    @price = <<Some External Query>>
  end

  def approve
    t = Transaction.find(params[:id])
    t.status = "Approved"
    t.update!
  end

Obviously this is not ideal. I don't want to query the API every time a user wants to submit a bid. Ideally, I could query this API every 5-10 seconds in the background and use the price in that manner. I have looked at a couple of techniques for running background jobs including delayed_job, sidekiq, and resque. For example in sidekiq I could run something like this:

  #app/workers/price_worker.rb
  class PriceWorker
    include Sidekiq::Worker

    def perform(*args)
      get_price
    end

    def get_price
        @price = <<Some External Query>>
      end
  end

  #config/initializers/sidekiq.rb
  schedule_file = "config/schedule.yml"

  if File.exists?(schedule_file) && Sidekiq.server?
    Sidekiq::Cron::Job.load_from_hash YAML.load_file(schedule_file)
  end


  #config/schedule.yml
  my_price_job:
    cron: "*/10 * * * * * "
    class: "PriceWorker"

That code runs. The problem is I am kind of lost on how to handle the price variable and pass it back to the user from the worker. I have watched the Railscasts episodes for both sidekiq and resque. I have written background workers and jobs that queue and run properly, but I cannot figure out how to implement them into my application. This is the first time I have dealt with background jobs so I have a lot to learn. I have spent sometime researching this issue and it seems like more background jobs are used for longer running tasks like updating db indexes rather than constantly recurring jobs (like an API request every 5 seconds).

So to sum up, What is the proper technique for running a constantly recurring task such as querying an external API in Rails? Any feedback on how to do this properly will be greatly appreciated! Thank you.

Answer 1

That is not how background jobs work. You're right, you have a lot of reading up to do. Think of running an asynchronous job processor like Sidekiq as running an entirely separate app. It shares the same code base as your Rails app but it runs completely separately. If you want these two separate apps to talk to each other then you have to design and write that code.

For example, I would define a cache with reader and writer methods, then have the cache populated when necessary:

1. someone loads product "foo" for the first time on your site
2. Rails checks the cache and finds it empty
3. Rails calls the external service
4. Rails saves the external service response to the cache using its writer method
5. Rails returns the cached response to the client

The cache would be populated thereafter by Sidekiq:

1. someone loads product "foo" for the second time on your site
2. Rails checks the cache and finds the cached value from above
3. Rails fires a Sidekiq job telling it to refresh the cache
4. Rails returns the cached response to the client

Continuing from step 3 above:

1. Sidekiq checks to see when the cache was last refreshed, and if it was more than x seconds ago then continue, else quit
2. Sidekiq calls the external service
3. Sidekiq saves the external service response to the cache using its writer method

When the next client loads product "foo", Rails will read the cache that was updated (or not updated) by Sidekiq.

With this type of system, the cache must be an external store of some kind like a relational database (MySQL, Postgres, Sqlite) or a NoSQL database (Redis, memcache). You cannot use the internal Rails cache because the Rails cache exists only within the memory space of the Rails app, and is not readable by Sidekiq. (because Sidekiq runs as a totally separate app)