Search code examples
dictionarydata-structureshashmaphashtableassociative-array

Is it always necessary to make hash table number of buckets a prime number for performance reason?


https://www.quora.com/Why-should-the-size-of-a-hash-table-be-a-prime-number?share=1

I see that people mention that the number of buckets of a hash table is better to be prime numbers.

Is it always the case? When the hash values are already evenly distributed, there is no need to use prime numbers then?

https://github.com/rui314/chibicc/blob/main/hashmap.c

For example, the above hash table code does not use prime numbers as the number of buckets.

https://github.com/rui314/chibicc/blob/main/hashmap.c#L37

But the hash values are generated from strings using fnv_hash.

https://github.com/rui314/chibicc/blob/main/hashmap.c#L17

So is there a reason why it makes sense to use bucket sizes that are not necessarily prime numbers?


Solution

  • The answer is "usually you don't need a table whose size is a prime number, but there are some implementation reasons why you might want to do this."

    Fundamentally, hash tables work best when hash codes are spread out as close to uniformly at random as possible. That prevents items from clustering in any one location within the table. At some level, provided that you have a good enough hash function to make this happen, the size of the table doesn't matter.

    So why do folks say to pick tables whose size is a prime? There are two main reasons for this, and they're due to specific cases that don't arise in all hash tables.

    One reason why you sometimes see prime-sized tables is due to a specific way of building hash functions. You can build reasonable hash functions by picking functions of the form h(x) = (ax + b) mod p, where a is a number in {1, 2, ..., p-1} and b is a number in the {0, 1, 2, ..., p-1}, assuming that p is a prime. If p isn't prime, hash functions of this form don't spread items out uniformly. As a result, if you're using a hash function like this one, then it makes sense to pick a table whose size is a prime number.

    The second reason you see advice about prime-sized tables is if you're using an open-addressing strategy like quadratic probing or double hashing. These hashing strategies work by hashing items to some initial location k. If that slot is full, we look at slot (k + r) mod T, where T is the table size and r is some offset. If that slot is full, we then check (k + 2r) mod T, then (k + 3r) mod T, etc. If the table size is a prime number and r isn't zero, this has the nice, desirable property that these indices will cycle through all the different positions in the table without ever repeating, ensuring that items are nicely distributed over the table. With non-prime table sizes, it's possible that this strategy gets stuck cycling through a small number of slots, which gives less flexibility in positions and can cause insertions to fail well before the table fills up.

    So assuming you aren't using double hashing or quadratic probing, and assuming you have a strong enough hash function, feel free to size your table however you'd like.