Search code examples
elasticsearchlogstashlogstash-configurationelasticsearch-query

Timezone causing different results when doing a search query to an index in Elastic Search

I'm trying to find out the results from a search query (ie: searching results for the given date range) of a particular index. So that I could get the results in a daily basis.

This is the query : http://localhost:9200/dialog_test/_search?q=timestamp:[2016-08-03T00:00:00.128%20TO%202016-08-03T23:59:59.128]

In the above, timestamp is a field which i added using my logstash.conf in order to get the actual log time. When i tried querying this, surprisingly i got a number of hits (total hits: 24) which should've been 0 since I didn't have any log records from the date of (2016-08-03) . It actually displays the count for the next day (ie: (2016-08-04), which has 24 records in the log file. I'm sure something has gone wrong with the timezone.

My timezone is GMT+5:30.

Here is my filtering part of logstash conf:

filter {        
grok {
        patterns_dir => ["D:/ELK Stack/logstash/logstash-2.3.4/bin/patterns"]
        match => { "message" => "^%{LOGTIMESTAMP:logtimestamp}%{GREEDYDATA}" }          
}
    mutate {
        add_field => { "timestamp" => "%{logtimestamp}" }
        remove_field => ["logtimestamp"]
}
date {
        match => [ "timestamp" , "ISO8601" , "yyyyMMdd HH:mm:ss.SSS" ]
        target => "timestamp"
        locale => "en"
}}

EDIT:

This is a snap of the first 24 records which has the date of (2016-08-04) from the log file:

enter image description here

And this is a snap of the JSON response I got when I searched for the date of 2016-08-03:

enter image description here

Where am i going wrong? Any help could be appreciated.

Solution

  • In your date filter you need to add a timezone

    date {
    match => [ "timestamp" , "ISO8601" , "yyyyMMdd HH:mm:ss.SSS" ]
    target => "timestamp"
    locale => "en"
    timezone => "Asia/Calcutta"       <--- add this
}