Search code examples
logstashlogstash-grok

ignore end part of log in grok

I am new with grok and logstash i have a log file which is separated with space like this

1477879888.908 728 486704579 TCP_REFRESH_UNMODIFIED/304 254 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/91.189.88.162 -

i just want to filler my log for only this part and ignore other part

1477879888.908 728 486704579 TCP_REFRESH_UNMODIFIED/304 254 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index

ignore other part (i just want 7 space separated data and ignoree other data

Solution

  • You can used this grok pattern.

    %{BASE10NUM:number1}%{SPACE}%{INT:number2}%{SPACE}%{INT:number3}%{SPACE}%{WORD:msg}/%{INT:number4}%{SPACE}%{INT:number5}%{SPACE}%{WORD:protocol}%{SPACE}%{URI:action}

    Input 

    1477879888.908 728 486704579 TCP_REFRESH_UNMODIFIED/304 254 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/91.189.88.162 -

    Output

    number1     477879888.908
number2     728
port    
number5     254
number4     304
msg         TCP_REFRESH_UNMODIFIED
action      http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index
protocol    GET
number3     486704579

    You can then merge msg and number4 to obtain a new field tcpMsg. Finally you remove msg, number4 and port.

    mutate {
  add_field => {
    "tcpMsg" => "%{msg}/%{number4}"
  }
  remove_field => ["msg", "number4","port"]
}

    Hope this helps.