Logstash Grok Pattern for ping

Question

I have this sample from a log:

Tue Mar 27 06:51:48 2018 PING www.google.com (172.217.169.100) 56(84) bytes of data.
64 bytes from sof02s31-in-f4.1e100.net (172.217.169.100): icmp_seq=1 ttl=128 time=17.4 ms

--- www.google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 17.482/17.482/17.482/0.000 ms

I want to make a grok pattern for logstash and extract things like TIMESTAMP,IPV4,TTL , as well as the RTT values min/avg/max from last 2 lines.

This log is from a pinging script to the same IP every second or so. I guess I need a multiline pattern to take the values for each of those 6 lines at once?

Any Help would be great!!!

Thanks

Answer 1

You don't need a multiline if you use Oniguruma syntax to escape newline i.e. \n.

For instance, (?<newline>(.|\r|\n)*) could match all unnecessary data in your log between two paragraphs, which is,

" time=17.4 ms\n\n--- www.google.com ping statistics ---\n1 packets transmitted, 1 received, 0% packet loss, time 0ms\n"

Your final grok pattern would then look like this,

%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR} %{WORD:PING} %{HOSTNAME:host} \(%{IP:ip_address}\) %{DATA} ttl=%{INT:TTL}(?<newline>(.|\r|\n)*)rtt min/avg/max/mdev = %{NUMBER:min}/%{NUMBER:avg}/%{NUMBER:max}/%{NUMBER:mdev} ms

and it will produce following output,

{
  "DAY": [
    [
      "Tue"
    ]
  ],
  "MONTH": [
    [
      "Mar"
    ]
  ],
  "MONTHDAY": [
    [
      "27"
    ]
  ],
  "TIME": [
    [
      "06:51:48"
    ]
  ],
  "HOUR": [
    [
      "06"
    ]
  ],
  "MINUTE": [
    [
      "51"
    ]
  ],
  "SECOND": [
    [
      "48"
    ]
  ],
  "YEAR": [
    [
      "2018"
    ]
  ],
  "PING": [
    [
      "PING"
    ]
  ],
  "host": [
    [
      "www.google.com"
    ]
  ],
  "ip_address": [
    [
      "172.217.169.100"
    ]
  ],
  "IPV6": [
    [
      null
    ]
  ],
  "IPV4": [
    [
      "172.217.169.100"
    ]
  ],
  "DATA": [
    [
      "56(84) bytes of data. 64 bytes from sof02s31-in-f4.1e100.net (172.217.169.100): icmp_seq=1"
    ]
  ],
  "TTL": [
    [
      "128"
    ]
  ],
  "newline": [
    [
      " time=17.4 ms\n\n--- www.google.com ping statistics ---\n1 packets transmitted, 1 received, 0% packet loss, time 0ms\n"
    ]
  ],
  "min": [
    [
      "17.482"
    ]
  ],
  "BASE10NUM": [
    [
      "17.482",
      "17.482",
      "17.482",
      "0.000"
    ]
  ],
  "avg": [
    [
      "17.482"
    ]
  ],
  "max": [
    [
      "17.482"
    ]
  ],
  "mdev": [
    [
      "0.000"
    ]
  ]
}