I want a regular expression for using in fluentd for parsing nginx error logs.
The sample row is:
2024/04/15 09:06:29 [error] 3443790#3443790: *176070165 limiting requests, excess: 2.957 by zone "RequestLimitForCommonApi", client: 77.81.151.129, server: test.com, request: "POST /capi/session/forgot HTTP/1.1", host: "test.com", referrer: "https://test.com/"
I'm using the following format for matching log parameters:
format1 /^(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<error>.*), (?<client>.*), (?<server>.*), (?<request>.*), (?<host>.*), (?<referrer>.*)/
But some log rows have 'uptime' parameter and some of them don't have.
Now what regular expression should I use to also match the 'uptime' parameter value if exists?
Sample log row with uptime:
2024/04/15 02:01:32 [error] 3443790#3443790: *172976982 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 86.55.16.251, server: test.com, request: "POST /api/test HTTP/1.1", upstream: "http://127.0.0.1:30110/api/test", host: "test.com", referrer: "https://test.com/"
you can try this code to match client an request for example and if there was another phrase like "test" between them, it could be ignored.
client: (?[^,]+),( test: )?((?[^,]+),)? request: (?[^,]+)
as you noticed, "test:" and which comes after it is optional.