Search code examples
javascripthtmlweb-scrapingfirefox-addonweb-crawler

How to check whether the html has changed?

Apologies if that's the wrong place, but have no clue where to ask.

We are building Firefox addon that works on selected websites. Now, because those websites tend to change once in a while, I want to run once a day a javascript script that will check whether the specific part of code has been changed, and if it did, we would get some return message (ideally directly to my email).

What would be the easiest, simplest way to do that? I've read about webcrawlers, webscrapers, but they seem to be doing far more than we need. We basically want to run from our PCs connected to internet a small script that will do that simple check and send us email as soon as the change to the specific part of html body has been detected (or to be more specific - selected elements have not been found).

Any help greatly appreciated

Solution

  • Basically you might set up Google Spreadsheet to scrape pages' parts thru IMPORTXML function (here with an example) using xpath. Then you set up notifications in a spreadsheet: Tools -> Notification Rules enter image description here

    Now each time the scraping function (IMPORTXML) gets content that is different to previous one, spreadsheet should trigger a mail to you.

    As an off-the-shelf alternative I also might recommend some tools that track pages' and/or their parts'(!) changes including notifying by email.