can't parse login using promtail correctlyl

Question

Could anyone offer advice on how to configure promtail correctly to parse the json output from my logs? I haven't been able to figure out why promtail is not identifying the json properties and showing them in Grafana.

my logback appender, in logback.xml:

<appender name="json"
    class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
        <jsonFormatter
            class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
            <prettyPrint>true</prettyPrint>
        </jsonFormatter>
        <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
    </layout>
</appender>

promtail.yaml

        ...
        ...
        ...
 scrape_configs:
    - job_name: parse-logs
      pipeline_stages:
        - docker: {}
        - json:
            expressions:
              subscriptionType:
              AuthenticationSuccess:
              user:
              execution_lasted: execution_lasted
              time-to-pull(.ms): time-to-pull(.ms)
            source: message
        - json:
            expressions:
              logger: logger
              level: level
              timestamp: timestamp
              message: message
        - labels:
            level:
            timestamp:
        ...
        ...
        ...

logs:

...
{
  "timestamp" : "2022-11-24 09:33:03.361",
  "level" : "DEBUG",
  "thread" : "http-nio-8081-exec-3",
  "logger" : "com.vanitysoft.boundariesio.unitedstates.domain.dao.impl.PostalGrabberDAOImpl",
  "message" : "{\"time-to-pull(.ms)\":12,\"user\":\"unknown\",\"number-of-postals\":1}",
  "context" : "default"
}{
  "timestamp" : "2022-11-24 09:33:03.361",
  "level" : "DEBUG",
  "thread" : "http-nio-8081-exec-3",
  "logger" : "com.vanitysoft.boundariesio.unitedstates.domain.dao.impl.PostalGrabberDAOImpl",
  "message" : "{\"time-to-pull(.ms)\":12,\"user\":\"unknown\",\"number-of-postals\":1}",
  "context" : "default"
}

After editing promtail config I preform a rollup update...

kubectl edit configmap loki-promtail --namespace loki-stack
kubectl rollout restart deployment/lke-monitor-grafana

Update promtail parsing requires the logs to be written on the same line(no pretty print!)

<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
    <jsonFormatter          class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
    </jsonFormatter>
     <appendLineSeparator>true</appendLineSeparator>
    <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
</layout>

Answer 1

Assuming you correctly copied in your promtail.yaml, there is a formatting error: -docker: {} should be - docker: {}

---
scrape_configs:
  - job_name: parse-logs
    pipeline_stages:
      - docker: {}
      - json:
          expressions:
            subscriptionType: null
            AuthenticationSuccess: null
            user: null
            execution_lasted: execution_lasted
            time-to-pull(.ms): time-to-pull(.ms)
          source: message
      - json:
          expressions:
            logger: logger
            level: level
            timestamp: timestamp
            message: message
      - labels:
          level: null
          timestamp: null