Search code examples
phpxpathscreen-scraping

span value hidden when scraping

I am trying to grab the number of players online at pokerstars.co.uk using xpath:

//*[@id="ncPlayers"]

But the content of span[id=ncPlayers] is coming up blank, when I look at the source code of pokerstars.co.uk the content is also hidden, how to they do this and how to I get around it?

Solution

  • They probably fill in the span with javascript.

    Or they declare it statically in the javascript, or they are using javascript to request (with GET or POST request, by using jQuery or some other javascript library) it from the server. In the first case, you can scrape the javascript and extract the number. In the second case, you can perform the request yourself and parse the returned JSON or whatever data.

    Either way, you'll have to dig into the javascript source used on the website.

    Thanks to Brendan, I did visit the site and figured out that the site loads the JSON content from 

    http://www.psimg.com/datafeed/dyn_banners/summary.json.js