Search code examples
pythonjsondictionaryleafletscreen-scraping

Scraping GPS data from Leaflet map


I'm trying to get the data from a leaflet web map on this site: https://www.bundesnetzagentur.de/DE/Vportal/TK/Funktechnik/EMF/start.html

I've figured out that when I click on an icon it requests html data from this type of url: https://www.bundesnetzagentur.de/emf-karte/hf.aspx?fid=23302

however i cant find a way to match the ID from this url to any gps coordinates. There is also a file within a folder js.asmx calles "jscontent?set=gsb2021", this also sometimes references an object id.

Maybe someone can give me some pointers? I've done some scraping with requests in python and selenium but i cant seem to figure this one out.

Thanks,

Regards Patrick

I dug around but couldn't find a json or geojson file.


Solution

  • Open Firefox Network Monitor and observe that the HTTP POST request to https://www.bundesnetzagentur.de/emf-karte/Standortservice.asmx/GetAMSNAktiv yields an encrypted payload, such as "ZIgHiilmI+MDzJ+QbnxQ3fdy1FfOVLjOwYeXAK3Wtha1bHAWz+2Eaf18HurO7i8Ls2tT6nDI9T9IveV7nOTyZ1O8WniITHZ7wLkhQTQ7SnOQ0zrfCG/aIrrzOtF6YMJSZjWyzjq+OhvHNSmGKkzsQVPi6Immn9mJCKR+chcuqd4="

    In the Firefox JavaScript console invoke:

    > DecryptData("ZIgHiilmI+MDzJ+QbnxQ3fdy1FfOVLjOwYeXAK3Wtha1bHAWz+2Eaf18HurO7i8Ls2tT6nDI9T9IveV7nOTyZ1O8WniITHZ7wLkhQTQ7SnOQ0zrfCG/aIrrzOtF6YMJSZjWyzjq+OhvHNSmGKkzsQVPi6Immn9mJCKR+chcuqd4=")
    "[{\"Titel\":\"Messung Paulus van Husen Schule\",\"Lng\":7.10373,\"Lat\":52.0333,\"fID\":\"f2e821de-24ae-41d0-a818-37eb1dcff030\"}]" 
    

    The function DecryptData is defined in https://www.bundesnetzagentur.de/emf-karte/js.asmx/jscontent?set=gsb2021