I wrote a program that needs to handle a very large data with the following libraries:
So, I'm having memory problems (The program uses a LOT) and I was thinking maybe I can replace this libraries (with something that already exists or my own implementations):
So, three questions:
I ended up by changing boost::unordered_multimap
for std::unordered_map
of vector
.
boost::unordered_multimap
consumes more than two times the memory consumed by std::unordered_map
of vector
due the extra pointers it keeps (at least one extra pointer per element) and the fact that it stores the key and the value of each element, while unordered_map
of vector
only stores the key once for a vector that contains all the colliding elements.
In my particular case, I was trying to store about 4k million integers, consuming about 15 GB of memory in the ideal case. Using the multimap I get to consume more than 40 GB while using the map I use about 15 GB (a little bit more due the pointers and other structures but their size if despicable).