Writing parser with | parse kind=regex operator

Question

I`m trying to make a parser for FortiAnalyzer. I recieve log in next format (example from Fortinet documentation) : 2020-05-12 17:01:16 log_id=0001010018 type=event subtype=system pri=information desc="User login/logout successful" user="admin" userfrom="JSON(10.100.55.254)" msg="user 'admin' with profile 'Super_User' logout from JSON(10.100.55.254)" session_id=5108 adminprof="Super_User"

So variable=value . There are about 30-40 variables and they order may vary.

So i parse information in next way : | parse kind=regex (name_of_my_column) with * "log_id" * "=" Log_ID: string "type=" Type: string "subtype=" Subtype: string and so on.... When i write 17 variables in one query it gives me an error : parse: regex mode exceeded max allowed matching groups. actual = 17, limit = 16 When i start new line with | parse kind=regex function, previous variable (#16) has all information that comes after variable #16 Variable #17 has the right information. end of first query with | parse kind=regex and start of a second information in variable #16 and #17 Can someone please give an advice, what should i add to my query to see results in right way? I also tried to use |extend function, but i think, that is not right way in my case because variables can be in different positions in log. Will be thankful to any advice.

Answer 1

you can use the parse-kv operator.

for example:

print input = ```2020-05-12 17:01:16 log_id=0001010018 type=event subtype=system pri=information desc="User login/logout successful" user="admin" userfrom="JSON(10.100.55.254)" msg="user 'admin' with profile 'Super_User' logout from JSON(10.100.55.254)" session_id=5108 adminprof="Super_User"```
| parse-kv input as (log_id:string, type:string, subtype:string, pri:string, ['desc']:string, user:string, userfrom:string, msg:string, session_id:long, adminprof:string) with (pair_delimiter=' ', kv_delimiter='=', quote='"')
| project-away input

log_id	type	subtype	pri	desc	user	userfrom	msg	session_id	adminprof
0001010018	event	system	information	User login/logout successful	admin	JSON(10.100.55.254)	user 'admin' with profile 'Super_User' logout from JSON(10.100.55.254)	5108	Super_User