How to map location (lon, lat) in logstash to visualize in kibana?

Question

I have a csv file holding longitude and latitude for some of the records (otherwise it's " "). Now I want to use logstash 5.1.2 to ge the data into elasticsearch 5.1.2. I've written the following conf-file but the location field is still mapped to text.

input {  
      file {
            path => "/usr/local/Cellar/logstash/5.1.2/bin/data.csv"
            start_position => "beginning"
            sincedb_path => "/dev/null"
      }
}

filter {  
    csv {
        columns => ['logtime', 'text', 'user', 'country', 'location']
        separator => ","
    }
    date {
        match => ["logtime", "yyyy-MM-dd HH:mm:ss"]
        timezone => "Europe/London"
        target => "Date"
    }
    if [latitude] and [longitude] {
    mutate { convert => {"latitude" => "float"} }
    mutate { convert => {"longitude" => "float"} }
    mutate { rename => {"latitude" => "[location][lat]"} }
    mutate { rename => {"longitude" => "[location][lon]"} }
    }
}

output {
  elasticsearch { 
    hosts => ["localhost:9200"] 
    index => "twitter"}
}

What am I supposed to do to make the location field mapped as geo-point and be able to visualize the points on the map in Kibana 5.1.2? Thanks

Answer 1

You need to create a mapping that maps location to a geo_point. The easiest way to do that is with an index template so that when you start using time based indices, it will auto-create the mapping when a new index is created.

PUT /_template/twitter
{
  "order": 0,
  "template": "twitter*",
  "mappings": {
      "properties": {
        "location": {
          "type": "geo_point"
        }
      }
    
  }
}

Then delete your /twitter index and re-index your data.

The above template says that any index that gets created with the name twitter* will that have any _type's location field turned into a geo_point.

**NOTE: After ES 7.0 Above : types were removed and when creating a mapping it no longer accepts types which is a breaking change **