How to extract value from log with grok and logstash

Question

I must extract value from a log composed by row like this:

<38>1 [2017-03-15T08:45:23.168Z] apache.01.mysite.com event=login;src_ip=xxx.xxx.xxx.xxx\, xxx.xxx.xxx.xxx\, xxx.xxx.xxx.xxx;site=FE-B1-Site;cstnr=1454528;user=498119;result=SUCCESS 

For example with %{IP:source} I obtain only the first IP but, sometimes, I have 3 IP address. How I can extract all IP,'cstnr', 'result' and 'user' ?

Answer 1

Looks like you have a nested, delimited key-value format. First delimiter is ;, with each of those a key=value. Additionally, the values are delimited on ,'. You have enough grok to get the first IP address, but I suggest doing something a bit different:

1. Use grok to grab the entire string after your site-name.
2. Use the kv filter with field_split => ';', which will create fields named the same as your keys.
3. Use the csv filter on the src_ip address captured in the kv filter stage.
   • Use columns => [ cstnr', 'result', 'user' ] to get those fields named right.