Search code examples
elasticsearchelasticsearch-py

Elasticsearch won't update results

I'm using Elasticsearch 6.6.0 and elasticsearch-py 6.3.0. I put some data into Elasticsearch, then made this little function to return a list of all the values of a particular variable in Elasticsearch:

from elasticsearch import Elasticsearch
es = Elasticsearch
def get_prior_records():
    records = es.search(
        index="my_index",
        body={"query": {"match_all": {}}, "_source": ["var_i_want"]})
    return [item["_source"]["var_i_want"] for item in records["hits"]["hits"]]

The first time I run this function, it works perfectly and returns a list of all the instances of var_i_want in the DB. However, when I add more data to the DB (and confirm the data's there via Kibana) the function keeps returning the original list to me. I can't get a list that includes my new data.

What am I doing wrong? Why won't my search update its results?

Solution

  • Probable reason is that you are not paging.

    The default size of search hits is controlled by "size" attribute provided in search request body and is 10 by default.

    If you want to fetch next page of results, ask for that using size and from attributes in query api.