Search code examples
pythonseleniumflashbeautifulsoup

Trying to get links of an interactive map (Web scraping .swf)

i need to create a web scraper for this website

However I need to get the links for the counties, stored in the interactive map

Unfortunately, for some reason, their search engine doesn't provide all the results as the interactive map does.

My question: Could anyone tell me how to get all the links for all the counties, without manually accessing them?

Thanks

Solution

  • Technically you can use a decompiler to do this job.

    There are free (e.g.: ActionScript Extractor) and paid (e.g.: Sothink SWF Decompiler) tools out there.

    you can reference this answer

    Edit : Most swf content gets external records from either a .xml or .json file.

    Without decompiling and just using the browser's Developer Tools we can see that an xml file is indeed accessed (maybe it contains what you want) :

    http://www.allpetservices.co.uk/uk_ir_locator.xml.
    Put view-source: in front of the link to read it (if there's an error message).

    In that xml you want to extract the contents (the xyz) of each & every <link> xyz </link> tag. This will give you the links of every entry on the map.