Search code examples
logstash-groknetscaler

Time field for Netscaler logstash grok filter

I tried to parse Netscaler logs for Logstash with Grok. I found following filter online

filter {
    if "netscaler" in [tags] {
            grok {
                    break_on_match => true
                    match => [
                            "message", "<%{POSINT:syslog_pri}> %{DATE_US}:%{TIME} GMT %{SYSLOGHOST:syslog_hostname} %{GREEDYDATA:netscaler_message} : %{DATA} %{IP:source_ip}:%{POSINT:source_port} - %{DATA} %{IP:vserver_ip}:%{POSINT:vserver_port} - %{DATA} %{IP:nat_ip}:%{POSINT:nat_port} - %{DATA} %{IP:destination_ip}:%{POSINT:destination_port} - %{DATA} %{DATE_US:DELINK_DATE}:%{TIME:DELINK_TIME} GMT - %{DATA} %{POSINT:total_bytes_sent} - %{DATA} %{POSINT:total_bytes_recv}",
                            "message", "<%{POSINT:syslog_pri}> %{DATE_US}:%{TIME} GMT %{SYSLOGHOST:syslog_hostname} %{GREEDYDATA:netscaler_message} : %{DATA} %{IP:source_ip}:%{POSINT:source_port} - %{DATA} %{IP:destination_ip}:%{POSINT:destination_port} - %{DATA} %{DATE_US:START_DATE}:%{TIME:START_TIME} GMT - %{DATA} %{DATE_US:END_DATE}:%{TIME:END_TIME} GMT - %{DATA} %{POSINT:total_bytes_sent} - %{DATA} %{POSINT:total_bytes_recv}",
                            "message", "<%{POSINT:syslog_pri}> %{DATE_US}:%{TIME} GMT %{SYSLOGHOST:syslog_hostname} %{GREEDYDATA:netscaler_message} : %{DATA} %{INT:netscaler_spcbid} - %{DATA} %{IP:clientip} - %{DATA} %{INT:netscaler_client_port} - %{DATA} %{IP:netscaler_vserver_ip} - %{DATA} %{INT:netscaler_vserver_port} %{GREEDYDATA:netscaler_message} - %{DATA} %{WORD:netscaler_session_type}",
                            "message", "<%{POSINT:syslog_pri}> %{DATE_US}:%{TIME} GMT %{SYSLOGHOST:syslog_hostname} %{GREEDYDATA:netscaler_message}"
                    ]
            }
            syslog_pri { }

    }}

For the time field, it's %{TIME} GMT, I'm wondering what it means, and how can I use this extracted time field in date() filter?

Solution

  • %{TIME} GMT

    basically means match regular expression TIME (?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9]) followed by a space and the word "GMT".

    more on grok patterns > http://grokdebug.herokuapp.com/patterns#

    I think for your purpose instead of %{DATE_US}:%{TIME} GMT you can use

    <ns_syslog_timestamp>%{DATE}:%{TIME} %{WORD:ns_syslog_timezone} and

    date {
    match => [ "ns_syslog_timestamp", "MM/dd/YYYY:HH:mm:ss" ] 
    locale => en
  }

    more on date filter > https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html.

    -Siva