Grok pattern to match URIPATH with optional URIPARAM
I want to use Grok Pattern for filtering out this
172.20.20.88 - - [10/Nov/2018:23:49:31 +0700] "GET /id/profile.pl?user=285&device=Bg3tlX HTTP/1.1" 502 852 "-" "Go-http-client/2.0" "0.009"
I am using COMMONAPACHELOG
%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)
I have tried %{URIPATH:request} and %{URIPARAM:request}. The result of request is still /id/profile.pl?user=285&device=Bg3tlX. My expectation is /id/profile.pl.
My reference is https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns
Your %{NOTSPACE:request} matches any 1 or more non-whitespace chars before HTTP/1.1" 502 85... as NOTSPACE pattern is \S+. So, it matches the whole /id/profile.pl?user=285&device=Bg3tlX substring.
You cannot use just URIPATH or URIPARAM, because you still need to match the rest of the input. You have to use both, but make URIPARAM optional after URIPATH by enclosing it within an optional non-capturing group, (?:...)?.
So, replace %{NOTSPACE:request} with
%{URIPATH:request}(?:%{URIPARAM:requestparam})?
^^^ ^^
Demo at https://grokdebug.herokuapp.com/:
- regex replace all dots exept last one doesn't work
- Python split() function :: Need to split "int_32\n' " so that I get int_32 alone
- Understanding and Fixing the regex?
- Azure frontdoor compliant RegEx rule for /users/*
- "The Best Regex Trick" in Raku
- PHP replace string with values from array
- Regular Expression with Two Names: One With Middle Initial and One Without
- Why does apt-cache search find packages which do not match the given regular expression?
- How do I grep for all non-ASCII characters?
- Regular Expression (RegEx) for IPv6 Separate from IPv4
- python regex to get key value pair
- Regex pattern to match a string ends with multiple charachters
- convert string which contains sub string to dictionary
- how to read only last 24 hours from a multi dated log file and to grep specific pattern in python
- Ansible regex_findall is not giving expected results with negative search
- How to use Python Regex to match url
- python Regex for mixed alpha numerica data and special characters
- python - Wrong regex used?
- Split every occurrence of Key=Value pairs in a string where the value include one or more spaces
- How to match words in a string which contain at least one vowel (a,i,o,u,e) using regex python
- Regex - negative lookbehind for any character excluding pure whitespace
- Extract subtring using regex python
- Capturing any character between two specified words including new lines
- Split a string with multiple delimiter & create a separate list
- python regex lookbehind to remove _sublabel1 in string like "__label__label1_sublabel1"
- Replace An Exact Grouped Part in RegEx Python
- How to remove all emoji (unicode) characters from a string python
- re.sub replace with matched content
- Python Regex Expression that finds variable string between whitespace, punctuation and/ or string end
- How can I modify a python regular expression to check whether string elements exist in another string?