Search code examples
solrout-of-memoryfaceted-search

Solr - Faceted navigation on large index

I have an index containing 1.2 billion of documents (Solr 1.4.1). I want to enable faceted navigation on a field (int type) containg around 250 unique values.

I am getting Java heap space java.lang.OutOfMemoryError with the default method (facet.method=fc), while the enum method is very slow (but works).

What is the best approach for a given number of documents and unique values?

Updated:

So if I understand correctly:

  • Memory usage for faceting using fc method is: MaxDoc * 4bytes (the field type is int, 64bit JVM), so that is: 1118950216 * 4bytes = 4.1GB (aprox.)

  • Memory usage for faceting using enum method is: NumberOfUniqueValues * SizeOfBitset = 250 * (1118950216 / 8) = 32GB

Is this correct?

I will try again the FC method (and give more ram to solr).

Thanks!

Solution

  • You'll have to tune the JVM memory allocation settings and/or add more memory to the server; or alternatively look into sharding the index.