Search code examples
ruby-on-railsrubyweb-scrapingnokogiri

Got the right node with Nokogiri, but need to search further


I am using this.

doc = Nokogiri::HTML(open(url))
pic = doc.search "[text()*='hiRes']"

to get this script node:

<script type="text/javascript">
var data = {
'colorImages': { 'initial': 
[{"hiRes":"http://ecx.images-joes.com/images
/I/71MBTEP1W9L._UL1500_.jpg","thumb":"http://ecx.images-joes.com/images
/I/41xE2XADIvL._US40_.jpg","large":"http://ecx.images-joes.com/images
/I/41xE2XADIvL.jpg","main":{"http://ecx.images-joes.com/images
/I/71MBTEP1W9L._UX395_.jpg":[395,260],"http://ecx.images-joes.com/images
/I/71MBTEP1W9L._UX500_.jpg":[500,329],"http://ecx.images-joes.com/images
/I/71MBTEP1W9L._UX535_.jpg":[535,352],"http://ecx.images-joes.com/images
/I/71MBTEP1W9L._UX575_.jpg":[575,379]}

and the node keeps going from there..

But the only thing I need to pull out is the entire URL that contains the string. "UL1500" or the URL that follows "hiRes:".. ex. http://ecx.images-joes.com/images/I/71MBTEP1W9L.UL1500.jpg

I looked up the class that Nokogiri returns, and its a Nokogiri::XML::NodeSet

But I'm not sure how to interact with it in order to get what I need?

Thanks


Solution

  • I went from just using Nokogiri to a regex expression.. but ended up finding this and it worked like magic!!

    https://stackoverflow.com/a/5939906/4386626