Set query size when performing scroll request via Spring Data Elasticsearch Repositories

Question

I have a Spring Boot application and am using spring-data-elasticsearch-5.1.2 to create a scroll request to fetch large sets of "book"s from elasticsearch. The setup looks something like this:

public interface BookRepository extends ElasticsearchRepository<Book, String> {

   Stream<Book> findByAuthorName(String authorName);

}

// and client code invokes like
try (Stream<Book> books = bookRepository.findByAuthor(authorName)) {
   return books.toList();
}
catch (Exception e) {
   // handle
}

This does work, but by default the scroll request size is 500 (i.e. the generated query has size: 500). I'd like to be able to change this value to fetch larger result sets, but am unsure how to do so. I've tried updating the repository method to accept a Pageable and then invoking with a larger page size. E.g.

public interface BookRepository extends ElasticsearchRepository<Book, String> {
   // Add Pageable parameter
   Stream<Book> findByAuthorName(String authorName, Pageable page);

}

// Pass pageable with larger size
try (Stream<Book> books = bookRepository.findByAuthor(authorName, Pageable.ofSize(1_000))) {
   return books.toList();
}
catch (Exception e) {
   // handle
}

However, the generated query doesn't seem to be impacted.

{
  "_source": { ... },
  "from": 0,
  "query": { ... },
  "size": 500, <----- HOW TO CHANGE?
  "track_scores": false,
  "version": true
}

How can I change the query size?

Edit: Adding relevant docs about scrolling with repositories:

• https://docs.spring.io/spring-data/elasticsearch/docs/5.1.2/reference/html/#elasticsearch.scroll
• https://docs.spring.io/spring-data/elasticsearch/docs/5.1.2/reference/html/#repositories.query-streaming

Answer 1

In the current release (5.1) the Pageable is not taken into account for repository methods returning a Stream<E>, there always the default size of 500 is used.

I created an issue for this, it's implemented and merged and will be contained from 5.2-RC1 on.