Search code examples
elasticsearchkibanaelasticsearch-aggregation

Can Kibana visualizations aggregate by a meta field such as _size?


I'd like to visualize how our average elasticsearch document size is changing over time.

  1. We use ElasticSearch v7.1. We use the AWS ElasticSearch service and the Mapper Size Plugin is installed.
  2. I enabled the _size field on my index. ref
  3. In Kibana "Discover" I can find small documents using: _size: <900
    • Or I can query from the command line: curl -H 'Content-Type: application/json' -s http://es.example.com/logstash-2019.10.17/_search -d '{"query": {"range": {"_size": { "lt": 900 }}}}' | jq .

Now I want to create a Kibana visualization, for example a date histogram with the median _size, but Kibana "Visualize" won't let me select _size as the aggregation field. Is there a way to visualize the size of documents?


_size is a "meta-field".

_size The size of the _source field in bytes, provided by the mapper-size plugin.

Maybe "meta fields" aren't supported in Kibana?

I can aggregate by _size when not using Kibana "Visualize":

curl -H 'Content-Type: application/json' -s http://es.example.com/logstash-2019.10.17/_search -d '{
  "query": {
    "range": {
      "_size": {
        "gt": 10
      }
    }
  },
  "aggs": {
    "sizes": {
      "terms": {
        "field": "_size",
        "size": 10
      }
    }
  },
  "sort": [
    {
      "_size": {
        "order": "desc"
      }
    }
  ]
}' | jq .

Solution

  • Question is old but looks not answered. The solution is:

    • go to Kibana Index Patterns
    • search and edit the one that you use in Kibana to retrieve data from Elasticsearch index (e.g. logstash-*)
    • edit --> Add field
      • use emit(doc['_size'].value) as a value
      • use Bytes as a format
    • save it

    Now the _size value should be available as any other (_source) field at your Kibana visualizations/dashboards etc.