Build a failsafe if sunspot-solr is not available

Question

I am currently building a Rails 3.2 application that uses the gem sunspot to provide search functionality for my application. sunspot uses Apache Solr to perform fulltext indexing and searching.

def index
    @search = Article.search do
      fulltext params[:search]
      with(:published_at).less_than(Time.zone.now)
      paginate :page => params[:page], :per_page => 10
      facet(:published_month)
    end
    @articles = @search.results

    respond_to do |format|
      format.html # index.html.erb
      format.json { render json: @articles }
    end
end

Right now my code just performs a search every time someone hits the article index page and then renders the results. My concern is that solr goes down for some reason and my app dies with it. How could I implement a failsafe for this action that performs a basic Article.all whenever solr goes down?

I know that I could just rescue from the exception, but that would still mean that every request generates a connection attempt to solr. Is there a way to prevent this? (E.g. catching the exception once and then waiting 5 minutes before the app tries to reconnect to solr)

Answer 1

As a contributor to Sunspot and the cofounder of Websolr, I recommend using rescue_from with a low-ish timeout.

Where Sunspot is concerned, it looks like the option to specify a timeout was recently added to RSolr, the library Sunspot uses, so if this is a feature you need then you should look into contributing a pull request to Sunspot to pass a timeout to RSolr.

Generating a failing request to Solr shouldn't be that expensive if you have proper timeouts in place. If Solr is down, you want it to be timing out on opening the TCP connection. A healthy Solr server should open a TCP connection in an order of magnitude less than a second, and start sending data back in no more than a second or two for really expensive queries.

An open timeout of 0.1 seconds and data timeout of 5 seconds should be more than enough here.

The next best option here is to have some other local proxy between your application and Solr which can throttle requests when Solr is down. Perhaps a caching middleware, or Varnish. Certainly I'd see that as a more complicated approach, unless such a proxy or cache is already a part of your application's infrastructure.