Query with Hibernate Search and get all items without paging

Question

I'm using Hibernate Search in my Spring Boot project. The most of the time it's a perfect solution for my needs except in one case when I have to get all items selected by full text search query.

Is there a chance to avoid Hibernate Search / Lucene paging while using Hibernate Search? Something similar to Pageable.unpaged()? I don't want to set page size to Integer.MAX_VALUE as it's kind of hack. But if there is no better solution, I'll have to use it that way.

Answer 1

You generally shouldn't do fetch all hits in a single request, because there could be millions of hits and loading them all is likely to cause memory issues.

If you really want to do this regardless, you can fetch all results simply by not setting a limit on your query:

Hibernate Search 6:

Session session = /* ... */;
List<MyEntity> = Search.session( session ).search( MyEntity.class )
        .where( f -> /* ... */ )
        .fetchAllHits();

Hibernate Search 5:

Session session = /* ... */;
FullTextSession fullTextSession = Search.getFullTextSession( session );
Query luceneQuery = /* ... */;
FullTextQuery ftQuery = fullTextSession.createFullTextQuery( luceneQuery, MyEntity.class );
List<MyEntity> hits = ftQuery.list();

But in many cases it won't work or will perform poorly because the hits take too much memory.

Ideally, you should use scrolling instead.

Here is an example with Hibernate Search 6:

SearchSession searchSession = /*...*/;
try ( SearchScroll<MyEntity> scroll = Search.session( session )
        .search( MyEntity.class )
        .where( f -> f.matchAll() )
        .scroll( 20 ) ) { 
    for ( SearchScrollResult<MyEntity> chunk = scroll.next(); 
            chunk.hasHits(); chunk = scroll.next() ) { 
        for ( MyEntity entity : chunk.hits() ) { 
            // ... do things with the entity ...
        }

        totalHitCount = chunk.total().hitCount(); 

        entityManager.flush();
        entityManager.clear();
    }
}

Hibernate Search 5:

FullTextQuery ftQuery = /*...*/;
ScrollableResults scroll = ftQuery.scroll();
scroll.beforeFirst();
int i = 0;
while ( scroll.next() ) {
    ++i;
    MyEntity entity = scroll.get()[0];

    // ... do things with the entity ...

    // Flush/clear the session periodically to free memory
    if ( i % 100 == 0 ) {
        // Only necessary if you changed entities
        session.flush();
        // Only necessary if you changed indexed entities
        // Also, only necessary in Hibernate Search 5
        fullTextSession.flushToIndexes();

        session.clear();
    }
}