Problem with Rails, Webrick or PostgreSQL? Malformed HTML generated after DB query

Question

I am running Rails 3 on Ubuntu with PostgreSQL 8.4 and WEBrick.

It has been working fine, but when displaying a large table (hundreds of rows), something is going wrong. The browser shows that the page is still loading, but it looks like the result of the query is getting truncated or Rails is breaking. There are no errors in the log or console.

The 8th line below is malformed HTML. It occurs around the 140th row of the table. But it continues to display more rows after the ones below.

<tr class="from_hc"> 
  <td class="date_and_time">Jul 13, 2011 11:00 AM</td> 
  <td class="name">Kim Orange</td> 
  <td>PHYSICAL_ACTIVITY</td> 
  <td>text text</td> 
  <td>ok</td> 
  <td></td> 
  <td><a href="/messag/a></td>
</tr>
<tr class="from_hc"> 
  <td class="date_and_time">Jul 13, 2011 11:00 AM</td> 
  <td class="name">Tom White</td> 
  <td>PHYSICAL_ACTIVITY</td> 
  <td>text text</td> 
  <td>ok</td> 
  <td></td> 
  <td><a href="/messages/260/edit">Edit</a></td> 
</tr>

Is it a problem with the number of rows returned? Is there a config that needs to be set in Rails, PostgreSQL or WEBrick? Could it be an issue with SSL? What could cause this?

The same code and table works fine on Heroku, so I wonder if it is a WEBrick config issue...

I increased the "shared_memory" for Postgres, but it did not help.

Thanks!!!

Answer 1

It turns out that there was non UTF-8 (UTF-16?) characters in a few rows of the database. Specifically, those smart quotes, apostrophe and ellipsis that Microsoft Word creates. A user had cut and pasted from Word and into a web form.

I wonder if there is a way to force UTF-8 in PostgreSQL to protect against this in the future...