Search code examples
elasticsearchkibanaelastic-stackdevtoolsksqldb

Can't count records in an index on Elasticsearch


I used this query to count hit number(docs) between specific time range with 15 minutes interval but the output is wrong

GET /logs-iis.access-default/_search
{
  "size": 0,
  "query": {
    "range": {
      "@timestamp": {
        "gte": "2023-09-17T00:00:00.000", 
        "lte": "2023-09-18T23:59:59.999"
      }
    }
  },
  "aggs": {
    "requests_over_time": {
      "date_histogram": {
        "field": "@timestamp", 
        "fixed_interval": "15m" 
      }
    }
  }
}

The output is whici is wrong when i check from discovery and lens, how can i list doc number with 15min interval for a specific time range.

{
  "took": 626,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "requests_over_time": {
      "buckets": [
        {
          "key_as_string": "2023-09-17T00:00:00.000Z",
          "key": 1694908800000,
          "doc_count": 592
        },
        {
          "key_as_string": "2023-09-17T00:15:00.000Z",
          "key": 1694909700000,
          "doc_count": 0
        },
        {
          "key_as_string": "2023-09-17T00:30:00.000Z",
          "key": 1694910600000,
          "doc_count": 0
        },
        {
          "key_as_string": "2023-09-17T00:45:00.000Z",
          "key": 1694911500000,
          "doc_count": 0
        },

... same for rest


Solution

  • This can be related to "size": 0.

    According to the official documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html

    size
    (Optional, integer) Defines the number of hits to return. Defaults to 10.
    
    By default, you cannot page through more than 10,000 hits using 
    the from and size parameters. 
    To page through more hits, use the search_after parameter.
    

    Here is another source:

    https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html