Temporarily I must use index.max_result_window to page over 10000 elements. I cannot find information on this - when I set index.max_result_window on some index do I have to close and open this index after setting index.max_result_window? And if not what changes requires reopening the index ?
And additionaly, if I have 20000 documents and I know that there will be no more than 100000, would it be better to set it to 100000 at the beggining or when document is indexed, count how many of them are in index and set it to 20000, 20001, 20004 etc. Whould it be worse for performance when it is set to 100000 in the beginning ?
index.max_result_window is a dynamic index setting that you can change at anytime without having to close/open your index, just run this:
PUT index/_settings
{
"index.max_result_window": 20000
}
it doesn't matter how many times you change it, but it's better to change it once you know how many documents you want to paginate over. If you know you want to paginate to 25000 docs, just increase the settings once, then paginate and set it back.
That being said, if you think of going higher, such as 100K, it might harm the performance of your cluster as it involves deep pagination, which is not good. In that case, and depending on your exact use case (i.e. why you want/need to paginate), you should prefer using search_after, with or without Point-In-Time (PIT).