Search code examples
osrmstxxlswapfile

OSRM extract and best swap stxxl location on 2 HDs

I want to set up the car profile data, but it is now taking 3 days and still at 100% Graphs (so way more to go) and I am thinking, maybe my swap and stxxl is just set up inappropriately.

I have the following system

  • 2HDs 4 TB SATA 6 Gb/s 7200 rpm HDD Enterprise-Class
  • 64 GB DDR4 RAM
  • Intel® Core™ i7-6700 Quad-Core Skylake incl. Hyper-Threading

HD1 has the

  • system
  • programs
  • osm.pbf file on it
  • the extracted data must go on this one
  • partition with 32G swap

HD2 mostly is empty (but the space is going to be needed for some other big data)

My extract approach was to set up another swap of 300G on HD2 and stxxl of 300G on HD2. But it seems to be too slow. iotop has some reading and some writing activity on it, but nothing major. htop does not show any significant cpu usage. I start with 8 processes.

Working with/on

  • planet-latest.osm.pbf (37G)
  • OSRM 5.9.0 (and now trying 5.12.0 docker directly from OSRM github, with no speed gain)

xxxxxxxxx

  1. Is there a better way to set up the additional swap and stxxl (maybe the swap on HD1 and only the stxxl on HD2)?

  2. If I could move the files and target locations of the extracted data as wanted, would it be even better to maybe do .pbf on HD1, all swap and stxxl on HD1 and extracted data on hd2(and then just copy it to HD1)? Thanks

    EDIT 4.9.2017 Seems like stxxl was disabled a few version back and needs to be enabled manually. So this might give me some boost. Yet I am still interested in the best split across two HDs.

Solution

  • The bottleneck here will not be STXXL. We disabled STXXL usage by default starting in 5.9. Looking at your server specs the culprit is the RAM. 64GB is not going to be enough for the whole planet.

    I think you would need about 180GB of RAM if you want to process the planet in one go. You can of course try to increase your swap space, but it will be very very slow.