C++ program speeds up when re-executed

Question

I am currently doing a performance testing of a C++ program. I need to bulk insert to a std::unordered_map or some other similar open source structure. I am inserting 30-40 char strings as key-value pairs and noticed an interesting behaviour. When code is first executed (Clion), it took 20 seconds to complete, I repeated the test (changing nothing), it was 14s, then again 8s, 4s, the code now runs around 3s. I should repeat, the only expensive operation in my code is bulk unordered_map insert, multithreaded with std::lock_guard<std::mutex>.

Another information that is important is that I am reading these keys and values from files. So I thought about some file caching happening in Ubuntu. But I was doing the same thing with different mechanisms and never experienced such a thing. Then I thought about some RAM allocation trick that may be keeping most of the map intact after the program is ended. But I am doing nothing to make that happen.

Why is this happening ? More than that, how can I reset this ? I need to do objective tests because my code will run in multiple servers without any pre-caching what so ever.

Thanks.

Answer 1

The first time you run the code it has to:

• Wake up the disk if it's spun down or idle
• Read the data off disk
• Read the executable off disk
• Possibly load in some shared libraries as well

The second time you run the code it can:

• Read the data from the OS read-cache in memory
• Read the executable from the OS read-cache in memory
• Read the shared libraries from the OS read-cache in memory
• Throttle up the CPU to a higher frequency now that it's under load

This is why it's important to run tests multiple times and for an extended period of time, not just seconds but at least 10-15 minutes continuously.

Your execution times will vary dramatically at first, but will converge to more consistent values later. You can always sort by run-time and find the 95th percentile, that is the time the program runs in 95% of the time.