TL;DR at bottom
I have a custom generated log file that I made that lists commands users run (along with some other thing). For that, I have got a grok script that successfully parses the message into fields. I've started encountering errors when I was asked to add some really old servers that generated different timestamp.
This is the the log on the new servers:
[2020-07-21 12:59:31] SERVER-DB-230 john:USER=root PWD=/root PID=[22714] CMD="echo test9" Exit=[0] CONNECTION=
[2020-07-21 12:59:33] SERVER-DB-230 john:USER=root PWD=/root PID=[22714] CMD="echo test10" Exit=[0] CONNECTION=
[2020-07-21 12:59:35] SERVER-DB-230 john:USER=root PWD=/root PID=[22714] CMD="clear" Exit=[0] CONNECTION=
This is the log on the old server (different timestamps):
Jul 21 13:02:53 SERVER-DEV-NEW-167 root: USER=root PWD=/root PID=[10638] CMD="echo 2" Exit=[0] CONNECTION=1.2.3.4
Jul 21 13:02:54 SERVER-DEV-NEW-167 root: USER=root PWD=/root PID=[10638] CMD="echo 3" Exit=[0] CONNECTION=1.2.3.4
Jul 21 13:02:56 SERVER-DEV-NEW-167 root: USER=root PWD=/root PID=[10638] CMD="echo 4" Exit=[0] CONNECTION=1.2.3.4
Since these are the syntax of logs I have, I think it's best to have an 'if' statement that says - if grok failed to parse, try parsing it with this grok code. Thing is, even though they're very similar, I wasn't able to make grok parse that data. I'm trying to get it working with the grok debugger but I just can't get it to work.
This is my current .conf in logstash: https://pastebin.com/QZv7zM1x
Does anyone know how to parse the second block of code into fields? And how to make it parse only if the first one failed? Thanks ahead!
TL;DR: need help parsing the second block of logs and have it parsed by grok only on failure
Here's a really lazy solution for the second block; if you need it to be more efficient then LMK.
(?<ts>%{SYSLOGTIMESTAMP}) (?<hname>(\b[\w\-]+\b)) (?<loggedas>%{WORD}): USER=(?<user>%{WORD}) PWD=(?<pwd>(\/[\w]+)) PID=(?<pid>(\[[\d]+\])) CMD="(?<cmd>[\s\S]+)" Exit=(?<exit>(\[[\d]+\])) CONNECTION=(?<connection>([\d]+\.[\d]+\.[\d]+\.[\d]+))
As another user mentioned, you can have it try multiple patterns.
The syntax would be,
grok { match => { "message" => [ "pattern1", "pattern2", "patternN" ] } }