Search code examples
phpxpathweb-scrapingdomxpath

Scrape Text With PHP & Display On Website

I am a complete beginner with PHP. I understand the concepts but am struggling to find a tutorial I understand. My goal is this:

  1. Use the xpath addons for Firefox to select which piece of text I would like to scrape from a site
  2. Format the scraped text properly
  3. Display the text on a website

Example)

// Get the HTML Source Code
$url='http://steamcommunity.com/profiles/76561197967713768';
$source = file_get_contents($url);

// DOM document Creation
$doc = new DOMDocument;
$doc->loadHTML($source);

// DOM XPath Creation
$xpath = new DOMXPath($doc);

// Get all events
$username = $xpath->query('//html/body/div[3]/div[1]/div/div/div/div[3]/div[1]');
echo $username; 
?>

In this example, I would like to scrape the username (which at the time of writing is mopar410).

Thank you for your help - I am so lost :( Right now I managed to use xpath with importXML in Google doc spreadsheets and that works, but I would like to be able to do this on my own site with PHP to learn how.

This is code I found online and edited the URL and the variable - as I am not aware of how to write this myself.

Solution

  • They have a public API.

    Simply use http://steamcommunity.com/profiles/STEAM_ID/?xml=1

    <?php

$profile = simplexml_load_file('http://steamcommunity.com/profiles/76561197967713768/?xml=1', 'SimpleXMLElement', LIBXML_NOCDATA);

echo (string)$profile->steamID;

    Outputs: mopar410 (at time of writing)

    This also provides other information such as mostPlayedGame, hoursPlayed, etc (look for the xml node names).