Converting nginx access log bytes to number in Kibana4

Question

I would like to create a visualization of the sum of bytes sent using the data from my nginx access logs. When trying to create a "Metric" visualization, I can't use the bytes field as a sum because it is a string type.

And I'm not able to change it under settings.

How do I go about changing this field type to a number/bytes type?

Here is my logstash config for nginx access logs

filter {
  if [type] == "nginx-access" {
    grok {
      match => { "message" => "%{NGINXACCESS}" }
    }
    geoip {
      source => "clientip"
    }
    useragent {
      source => "agent"
      target => "useragent"
    }
  }
}

Since each logstash index is being created as an index, I'm guess I need to change it here.

I tried adding

mutate {
  convert => { "bytes" => "integer" }
}

But it doesn't seem to make a difference.

Answer 1

Field types are configured using mappings, which is configured at the index level and can hardly change. With Logstash, as a new index is created everyday, so if you wan't to change these mappings either wait for the next day or delete the current index if you can.

By default these mappings are generated automatically by Elasticsearch depending on the syntax of the indexed JSON document and the applied Index Templates:

# Type String
{"bytes":"123"}

# Type Integer
{"bytes":123}

In the end there are 2 solutions:

• Tune Logstash, to make it generate an integer and let Elasticsearch guess the field type → Use the mutate/convert filter
• Tune Elasticsearch, to force the field bytes for the document type nginx-access to be of type integer → Use Index Template:

Index Template API:

PUT _template/logstash-nginx-access
{
  "order": 1,
  "template": "logstash-*",
  "mappings": {
    "nginx-access": {
      "properties": {
        "bytes": {
          "type": "integer"
        }
      }
    }
  }
}