Search code examples
grafanagrafana-loki

loki ingress-nginx label extraction via regexp logql

{namespace="ingress-nginx"} |= "gymauto-frontend" |~ "GET (/main)" | json

I get these entries

enter image description here

Now I need to extract the duration of the request from the "log" label, but I am not sure how to construction the logql query so I can do something like this

{namespace="ingress-nginx"} |= "gymauto-frontend" |~ "GET (/main)" | json | ...regexp () | duration > 10s

So I can filter only for queries GET /main* that are longer than 10s.

The log example:

10.110.1.132 - - [27/Aug/2021:02:37:06 +0000] "GET /main.d8e010115d6aba1bd049.js.map HTTP/2.0" 200 10842370 "https://gym.com/students/edit/173" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15" 34 20.031 [gymauto-gymauto-frontend-service-v1-8080] [] 10.110.5.217:8080 10842370 20.035 200 a7c4299aaa7ea674e91d13315
Solution

  • If you can use Loki v2.3 you should be able to use the newly introduced pattern parser expression.

    It will be easier to use this instead of the old regexp parser. In your example a pattern like:

    <_> - - <_> "<method> <path> <_>" <status> <_> "<_>" "<_>" <_> <duration> [<service>] [<_>] <_> <_> <_> <_> <_>

    should work. In this case we are extracting from the log the parts that we want to use: method, path, service, duration, etc. A query for filtering on the GET requests to the /main.* path on the gymauto-frontend service may look like:

    {namespace="ingress-nginx"} 
| pattern `<_> - - <_> "<method> <path> <_>" <status> <_> "<_>" "<_>" <_> <duration> [<service>] [<_>] <_> <_> <_> <_> <_>`

    If you inspect (in Grafana) the extracted log fields you should see something like:

    extracted log fields view in grafana

    You can notice, that only the named fields are extracted, the rest of the log line is ignored. Now we can filter on the specific entries that you are interested in:

    {namespace="ingress-nginx"} 
| pattern `<_> - - <_> "<method> <path> <_>" <status> <_> "<_>" "<_>" <_> <duration> [<service>] [<_>] <_> <_> <_> <_> <_>` 
| service=~".*gymauto-frontend.*"
| method="GET"
| path=~"/main.*" 
| duration > 10

    If you are using an older version of Loki then you will need to replace the pattern parser expression with regexp and define a regexp that matches your log entry.