Search code examples
rubyurihpricot

Hpricot error parsing special characters in URI

I'm working on a ruby script to grab historical stock prices from Yahoo, using Hpricot to parse the pages. This is mostly straighforward: the url is "http://finance.yahoo.com/q/hp?s=TickerSymbol" For example, to look up Google, I would use "http://finance.yahoo.com/q/hp?s=GOOG"

Unfortunately, it breaks down when I'm looking up the price of an index. The indexes are prefixed with a caret, such as "http://finance.yahoo.com/q/hp?s=^DJI" for the Dow.

The line:

ticker_symbol = '^DJI'
doc = Hpricot(open("http://finance.yahoo.com/q/hp?s=#{ticker_symbol}"))

throws this exception:

bad URI(is not URI?): http://finance.yahoo.com/q/hp?s=^DJI

Hpricot chokes on the caret (I think because the underlying Ruby URI library does). Is there a way to escape that character or force the library to try it?

Solution

  • The escape for ^ is %5E; you could do a straight substitution on the URL.

    http://finance.yahoo.com/q/hp?s=%5EDJI