Promtail and Grafana - json log file from docker container not displayed

Question

my application's services are deployed via docker-compose. Currently, I also deployed Grafana, Loki and Promtail within the same docker-compose network.

Following the getting-started guide, collecting and displaying the log files from /var/log with the config

- job_name: system
entry_parser: raw
static_configs:
- targets:
    - localhost
  labels:
    job: varlogs
    __path__: /var/log/*log

works fine.

However, my backend logs (NestJS) into a log file which is stored in a docker volume. Example log entry:

{"message":"Mapped {/api/drink, POST} route","context":"RouterExplorer","level":"info","timestamp":"2021-03-23T17:08:16.334Z"}

The path to the log is

/var/lib/docker/volumes/my_volume/_data/general.log

When I add the following config to Promtail

 - job_name: backend
pipeline_stages:
- json:
    expressions:
      level: level
      message: message
      timestamp: timestamp
      context: context
static_configs:
- targets:
    - localhost
  labels:
    job: backend
    __path__: /var/lib/docker/volumes/my_volume/_data/general.log

and use the query {job="backend"} in Grafana, nothing is displayed. Furthermore, the log of the promtail container doesn't give any information.

What am I missing?

Thank you in advance!

Answer 1

In your pipeline stages you need to store the extracted values:

pipeline_stages:
- json:
    expressions:
      level: level
      message: message
      timestamp: timestamp
      context: context
- timestamp:
    source: timestamp
- labels:
    level:
    context:
- output:
    source: message

This will set the timestamp, the labels context, level and the message will be the log line.

Documentation can be found here.