Mechanize::ResponseReadError - Content-Length does not match response body length

Question

I have encountered this error while requesting a page with Mechanize:

Mechanize::ResponseReadError
Content-Length (17317) does not match response body length (17070) (Mechanize::ResponseReadError)

Any thoughts on why this occurs and how I could get about fixing it are much appreciated!

Answer 1

It happens that sites return a wrong content length-value. Catch the error and force page parsing.

agent = Mechanize.new
begin
  page = agent.get 'http://bad.com'
rescue Mechanize::ResponseReadError => e
  page = e.force_parse
end

You can also set agent.ignore_bad_chunking to true — but then beware of possible silent content loss.