Whats the default size of a Vec::new() and when does it reallocate if it exceeds the default size? I'm seeing some performance issues when working with vectors (I think this is the main reason) once they grow enough. The vector has Vec<u8> values, so, the resultant vector is like the following
let items: Vec<Vec<u8>> = Vec::new();
I'm doing massive insertions just to see if it's capable of handling millions of values, but as soon as it gets to ~30k values, the insertion process tends to double the time, and as time goes by, it will double again.
I want to store some identifiers within the vector in order to later on implement some pagination on the vector for an API, but I think using unbounded vectors is causing this issue, I'm not sure.
Besides the usage of vectors, I'm also using a HashMap at the same time to insert the Vec<u8> key and some object as value, so HashMap can also be the issue.
whats the default size of a
Vec::new()
Zero:
The vector will not allocate until elements are pushed onto it.
when does it reallocate if it exceeds the default size
It is unspecified, but currently the growth rate is 2x the previous capacity (starting from 1/4/8 depending on the element size). That is, the vector doubles its size every time you push a new element and it has no capacity left. This is usually a good growth rate, but if you insert a lot of elements you can indeed end with very bad performance and memory usage because you may allocate roughly double the memory you need and the last reallocations can require very large copies.
Note that not every time you grow one-by-one - for example, if you extend() by an iterator (or collect() an iterator), the iterator may know its exact size and allocate only once.
If you know the required number of elements ahead of time, you can call with_capacity() or reserve_exact() to ensure the vector will allocate only once.