Search code examples
grafanagrafana-loki

Promtail error pipeline stage must only contain one key


When trying to setup Promtail I get the following error:

level=error ts=2020-11-27T06:10:30.310583Z caller=main.go:104 msg="error creating promtail" error="failed to make file target manager: pipeline stage must contain only one key"

I am running the following from the command line.

promtail-windows-amd64.exe --config.file=../conf/promtail-local-config.yml

My log lines looks like this:

13:21:03.183 - INFO - Successfully received document from '127.0.0.1'. Saved as 'c:\test\test_file.txt' in 102ms from address '/127.0.0.1'

13:21:05.275 - WARN - Failed to receive document from '127.0.0.1'. Error creating file c:\test\error_file.txt'

My config looks as follows:

scrape_configs:
- job_name: promtailTest
  pipeline_stages:
  - match:
    selector: '{job="promtailTest"}'
    stages:
    - regex:
      expression: '^(?P<timestamp>\\d{2}:\\d{2}:\\d{2}\\.\\d{3})\\s\\-\\s(?P<logLevel>[A-Z]{4,5})\\s\\-\\s(?P<logMessage>.*)$'
    - labels:
      logLevel:
  static_configs:
  - targets:
      - localhost
    labels:
      job: promtailTest
      app: promtailTest
      host: LOCAL
      __path__: C:/test/logs/*log

When I take out the pipeline_stages: section then I do see the lines in grafana however I cannot get the regex part to work. I actually want to add a label for the logging Level (so I can count the errors)


Solution

  • BAH!! I am a fool. I think the problem was with my format (YAML can be a real PITA!!!)

    My new config now looks like this and it works.

    scrape_configs:
    
      - job_name: promtailTest
        static_configs:
        - targets:
            - localhost
          labels:
            job: promtailTest
            app: promtailTest
            host: APOLLO99
            __path__: D:/SXI/XPress/Dispatch/logs/*log
        pipeline_stages:
        - match:
            selector: '{job="promtailTest"}'
            stages:
            - regex:
                expression: "^^(?P<myTime>\\d{2}:\\d{2}:\\d{2}\\.\\d{3})\\s\\-\\s(?P<logLevel>[A-Z]{4,5})\\s\\-\\s(?P<logMessage>.*)$$"
            - labels:
                logLevel: