How big can Erlang DETS be and what to do if its too small?

Question

All I need is a large persistent lookup table in Erlang and dets seems like just the thing though I need a definative answer to:

• just how big the total size of the binaries in the table can be.
• how big each entry can be
• what to do if the answer to the first question is less than 100G

Answer 1

One obvious approach, once it is thought of, is to hash dets entries over multiple dets files.

A linear hash should make it dynamically growable, by splitting buckets into newly created dets files when one file reaches an upper threshold.

There are also a number of port drivers that enable you to use sleepycat/berkely db, or tokyo tyrrant. Those databases have file limits that are much higher than 2Gb.