Search code examples
logginggrafanagrafana-lokipromtail

Is there a way to use Loki to query logs from MySQL database?


Ok. We have Promtail to gather text log files and send them to loki.
Also I can create MySQL data source to query data from database for visialization purposes.

I am curious. Is it possible to configure Loki/Promtail or there is some logging driver which allows Loki to access them.

Why I am asking about this. Because our legacy system stores logs of our application into database into (ID, time, text) fields. So I am trying to find a way to import these logs into Loki or to configure Promtail to fetch logs not from text files but from this database table.

Is that possible?


Solution

  • This is possible.

    Found how to post log to Loki directly:

    /loki/api/v1/push is the endpoint used to send log entries to Loki. The default behavior is for the POST body to be a snappy-compressed protobuf message. Alternatively, if the Content-Type header is set to application/json, a JSON post body can be sent in the following format:

    {
      "streams": [
        {
          "stream": {
            "label": "value"
          },
          "values": [
              [ "<unix epoch in nanoseconds>", "<log line>" ],
              [ "<unix epoch in nanoseconds>", "<log line>" ]
          ]
        }
      ]
    }
    

    Here is blog post how to do that with python: https://medium.com/geekculture/pushing-logs-to-loki-without-using-promtail-fc31dfdde3c6#8290