Search code examples
guavabloom-filter

How to get the memory size of Guava's BloomFilter

I am unable to find the direct API (i.e. total bits, memory size in bytes should be total_bits / 8). The only way I found is to serialize into a byte array, but this may take further memory:

    BloomFilter<String> bloomFilter = BloomFilter.create(Funnels.
            stringFunnel(StandardCharsets.UTF_8), 100_000_000);
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    bloomFilter.writeTo(baos);
    System.out.println(baos.toByteArray().length);

Is there any efficient ways to do this?

Solution

  • You can't do it easily with the Guava Bloom filter API. (Other libraries such as FastFilter have methods such as getBitCount() so you can easily retrieve it. But the Guava Bloom filter currently, has no such method.)

    There are online tools that allow you to calculate the missing parameters for Bloom filters, for example the Bloom Filter Calculator. As the Guava Bloom filter is a regular Bloom filter, you can calculate the space usage yourself from the parameters, using the formulas of the Bloom filter (which the Guava Bloom filter implementation also uses internally):

    import static java.lang.Math.log;
import static java.lang.Math.pow;

// expected insertions
int n = 100_000_000;

// false positive probability; default: 3% for Guava
double fpp = 0.03;

double bits = n * log(fpp) / -log(pow(2, log(2)));
int bytes = (int) Math.ceil(bits / 8);

// result for the above: 91230511

    This is only an estimation; the number could be off due to rounding. And of course, serializing a Java object uses a bit more space, and heap memory.