Search code examples
grafanagrafana-loki

Loki display log message and extra fields separately

In https://github.com/grafana/loki/issues/4249 I found interesting screenshot. enter image description here

On this screenshot I see that log level and message are displayed bold with white text and other metadata (collected from log message) displayed on separate line with grey color.

I have searched docs and haven't found how it's possible to achieve that. To be honest I'm searching for something like "short message" in ELK to make developers see metadata only when they are actually needs it. Could you please point me to the doc how to achieve that please?

Solution

  • Short answer: I found that there isn't such UI function in Grafana UI. But there's two features that can help you achieve such result:

    1. Line formating - allows you to show only selected parts of message
    2. ANSI escape sequence rendering - that allows you to change font settings (bold/italic/color)

    Long answer:

    • Here's my initial test query (that shows only messages that have "HornetQ") {appname=~".+"} |= "HornetQ" it produces following output. initial log view
    • I have added line formatting to the query to show only message field by default
    {appname=~".+"} |= "HornetQ"
  | json
  | line_format "{{ .message }}"

    format message view But if you would open message details you would see all json fields anyway format message details view

    • Let's add modify line format to show preview of extra fields on separate line.

    We would use pattern '<_entry>' to save initial json for further processing. Also we would use gotpl loop in line_format and if that would skip message field

    {appname=~".+"} |= "HornetQ"
  | pattern `<_entry>` 
  | json
  | line_format "{{ .message }}\n{{ range $k, $v := (fromJson ._entry)}}{{if ne $k \"message\"}}{{$k}}: {{$v}} {{ end }}{{ end }}"

    log view with fields preview on separate line

    • Let's make our messages better readable by changing font options.

    To achieve that we would use ANSI escape sequences (additional info)

    {appname=~".+"}
  | pattern `<_entry>` 
  | json
  | line_format "\033[1;37m{{ .message }}\033[0m\n{{ range $k, $v := (fromJson ._entry)}}{{if ne $k \"message\"}}\033[1;30m{{$k}}: \033[0m\033[2;37m{{$v}}\033[0m {{ end }}{{ end }}"

    colorize log message and extra fields preview You can see that |= "HornetQ" part is missing in last query, that because it breaks last query (with colouring), so I skip it.

    P.S. So for now my solution doesn't work with fulltext search :(