Search code examples
pythonparsingbigdataopenstreetmap

How to handle very big OSMdata with Pyrosm


I use Pyrosm for parsing *.osm.pbf files.
On their websites it says "When should I use Pyrosm? However, pyrosm is better suited for situations where you want to fetch data for whole city or larger regions (even whole country)."

However when I try to parse to big .osm.pbf files, I get memory problems. Is there a solution for that, e.g. like chunking in pandas? Or do I need to split up the file, if yes, how?


Solution

  • So my solution is to split the files up via osmium-tools.