Logstash uses current year for timestamp without year and shows events with future dates

I have logs wherein there's no year mentioned, and while parsing, logstash adds current year and makes the event appear in future dates.

i checked on forums for a similar issue but unfortunately, there wasn't any solution for it. Based on the discussions, the date filter should automatically put the correct year but its not happening in my case. Below is a sample of log timestamp.

Tue Oct 20 11:04:30.996 data: zone1 data: zone2 this is log date from Oct 2020.

And the output i get is

          "new" => 2021-10-19T08:04:30.996Z,
      "logtime" => "Tue Oct 20 11:04:30.996",
         "path" => "/home/zone.log",
      "message" => "Tue Oct 20 11:04:30.996 data: zone1 data: zone2",

Grok/date filter is:

grok
        {
        match => { "message" => "(?<logtime>%{DAY} %{MONTH} %{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) ........" }
        }

        date
        {
        match =>  [ "logtime", "EEE MMM dd HH:mm:ss.SSS"  ] 
        target => "new"
        }

There's no dateparsefailure but the timestamp goes into future. Please advise on this.

Solution

The date filter cannot possibly know the correct year in all cases. An issue for this has been open on github for several years and there is much discussion there about how to fix it. However, I do not expect it will be addressed.

The parser has some heuristics to guess the right year:

If the timestamp is from December and the current date is in January, assume it is from last year.
If the timestamp is from January and the current date is in December, assume it is from next year.
Otherwise use the current year.

If that logic does not work for you then you would have to implement your own logic, probably in a ruby filter, to decide which year to add to the timestamp before parsing it with a date filter.

If you just want to delete events in the future you could use

ruby { code => 'if Time.now.to_f < event.get("@timestamp").to_f ; event.cancel ; end' }