Hibernate search lucene consuming database connections

Question

I have configured maxPoolSize 10 for c3p0 pooling. minPoolSize is 3. Whenever i start my server and execute show processlist for mysql database, there are three connections established. This will be remain three only unless there is huge traffic to the server. This is fine.

I have implemented search functionality which starts indexing for the first time. When application does search and when i check show processlist for my database , all the db connections are open now. I see all the 10 connections are open. Is it the accepted behavior. Ideally i would say utilizing opening 2 or 3 connections could have been fine. Why hibernate lucene is utilizing many connections? Can I configure the max connections somewhere for lucene.

I am posting my code below.

fullTextEntityManager.createIndexer().startAndWait(); 

will execute only one time for the application

    @Transactional
public List<E> searchRecord(String searchText) {

    FullTextEntityManager fullTextEntityManager = org.hibernate.search.jpa.Search.getFullTextEntityManager(em);

    if (CREATE_INDEX.equals("START")) {
        try {
            log.info("creating indexes");
            fullTextEntityManager.createIndexer().startAndWait();
            CREATE_INDEX = "END";
        } catch (InterruptedException e) {
            log.info("error in creating indexes", e);
        }
    }
    log.info("lucene query builder");
    QueryBuilder qb = fullTextEntityManager.getSearchFactory().buildQueryBuilder().forEntity(getEntityClass())
            .get();
    // org.apache.lucene.search.Query luceneQuery =
    // qb.keyword().onFields("name")
    // .matching(searchText).createQuery();
    if(searchText.contains(" ")){
        String[] splittedText = searchText.split(" ");
        searchText = splittedText[0];
    }
    org.apache.lucene.search.Query luceneQuery = qb.keyword().wildcard().onField(getSearchField())
            .matching(searchText.toLowerCase() + '*').createQuery();

    javax.persistence.Query jpaQuery = fullTextEntityManager.createFullTextQuery(luceneQuery, getEntityClass());

    List<E> result = jpaQuery.getResultList();
    return result;
}

Answer 1

The purpose of the mass indexer is to process your whole database (or at least the parts that should be indexed by Hibernate Search) to rebuild the index. As you can expect, this is a highly resource-intensive process that requires lots of database accesses, and executing those accesses in parallel can speed up the process significantly. That's why it uses many connections; by default it uses 7 connections.

Now, if you want the mass indexer to use less connections, you can simply ask the mass indexer to use fewer threads:

fullTextEntityManager.createIndexer()
    .threadsToLoadObjects(2)
    .startAndWait();

The above will only use 3 threads: 2 to load entities from the database, and 1 to push the documents to the index. Obviously reindexing will be slower, though.

See this section of the documentation for more information on this topic.