Initial block size for memory pool

Question

I am implementing a memory pool class using C++ template, and I am wondering what a good size could be for the block. For example:

template <typename T>
class Mempool {

  unsigned char* block;

  // constructor. 
  Mempool() {
    block = malloc(sizeof(T)*DEFAULT_N)
  }

};

In the above example, the block size actually depends on the type T and a default value of the number of elements to be created. What is the best (or common) practice to do this? Should I consider memory alignment for block size here?

Answer 1

I can only give some general advice as a lot will depend on the actual use case:

Alignment: I guess the pool should make sure that objects in it are correctly aligned. That means you may want to place objects at memory locations that are at least a multiple of std::alignment_of<T>::value or alignof(T).

Cache-friendliness: The pool may also round up the object locations to multiples of cache-line sizes, so (small) objects will never be placed across two cache-lines but always be located in a single one.

Padding: If objects are very small (only a few bytes) but you have very many of them, then any extra padding might increase memory requirements much, and it depends on the application whether this will be a problem or not. Too much padding may actually hurt performance when objects are not held in close-together locations and there are lots of cache misses.

Underlying memory management: And finally, a good block size may also depend on the underlying storage. You may want your allocation sizes to match OS page sizes, or use multiples of them. Additionally, if malloc is expensive, then you probably want to call it as few times as possible in order to avoid syscall overhead and contention. In environments where malloc is cheaper, it may be ok to use smaller initial block sizes and malloc more often, though malloc will still have some overhead. A great deal of this is OS-specific.

So whatever initial sizes you pick, you should measure memory usage and allocation performance for some typical workloads. This will allow you to make more educated estimates than pure guessing can provide.

Finally, there are ready-to-use malloc-replacements such as tcmalloc, jemalloc, which may have completely different characteristics than your OS's malloc. A good option is therefore to check one of these instead of rolling your own allocator.