Is there a reason there is inconsistent behavior when doing a search on an elastic index after it is reindexed?

Question

I am using the elasticsearch python API and when doing a search on an index that was recently reindexed I see some behavior where I sometimes get results, and sometimes don't. The only way to reproduce this behavior is running the code over and over again until failure, and the same result being seen.

es.reindex({"source": {"index": "initial-index"},
            "dest": {"index": "dest-index"}
            }, 
    wait_for_completion=True, refresh=True)

dest_index_results = es.search(index="dest-index", size=100)
hits = get_hits(dest_index_results['hits']['hits'])

The dest_index_results and hits would come out to be empty. Any help understanding why would be highly appreciated.

Answer 1

there is a config in index setting called refresh_interval . base on the time you set in this config, the docs will be searchable. for example if the time is set to 1 minute, the docs will be ready for search , one minute after indexing.

you can change this setting with below command;

PUT /twitter/_settings
{
    "index" : {
        "refresh_interval" : "10s"
    }
}