In RoR, how do I catch an exception if I get no response from a server?

Question

I’m using Rails 4.2.3 and Nokogiri to get data from a web site. I want to perform an action when I don’t get any response from the server, so I have:

begin
  content = open(url).read
  if content.lstrip[0] == '<'
    doc = Nokogiri::HTML(content)
  else
    begin
      json = JSON.parse(content)
    rescue JSON::ParserError => e
      content
    end
  end
rescue Net::OpenTimeout => e
  attempts = attempts + 1
  if attempts <= max_attempts
    sleep(3)
    retry
  end
end

Note that this is different than getting a 500 from the server. I only want to retry when I get no response at all, either because I get no TCP connection or because the server fails to respond (or some other reason that causes me not to get any response). Is there a more generic way to take account of this situation other than how I have it? I feel like there are a lot of other exception types I’m not thinking of.

Answer 1

This is generic sample how you can define timeout durations for HTTP connection, and perform several retries in case of any error while fetching content (edited)

require 'open-uri'
require 'nokogiri'

url = "http://localhost:3000/r503"

openuri_params = {
  # set timeout durations for HTTP connection
  # default values for open_timeout and read_timeout is 60 seconds
  :open_timeout => 1,
  :read_timeout => 1,
}

attempt_count = 0
max_attempts  = 3
begin
  attempt_count += 1
  puts "attempt ##{attempt_count}"
  content = open(url, openuri_params).read
rescue OpenURI::HTTPError => e
  # it's 404, etc. (do nothing)
rescue SocketError, Net::ReadTimeout => e
  # server can't be reached or doesn't send any respones
  puts "error: #{e}"
  sleep 3
  retry if attempt_count < max_attempts
else
  # connection was successful,
  # content is fetched,
  # so here we can parse content with Nokogiri,
  # or call a helper method, etc.
  doc = Nokogiri::HTML(content)
  p doc
end