Search code examples
dockerlogginggoogle-compute-engineslf4jstackdriver

Why isn't Stackdriver capturing log levels from Slf4j logs from Compute Engine Instance running a Docker Image?


I currently have a gradle Spring Boot app running as a Docker image in a GCP Compute Engine instance. In my Application I added the lombok @Slf4j annotation and in the main method I added the line log.info("Hello world"); and ran the image in my GCE instance via docker run -d --rm -it -p 8888:8080 {image} and checked the Stackdriver logs.

I would expect to be able to filter via log level (INFO, WARNING, etc.), but it seems that the logs are not mapping the log level appropriately, meaning they only show up when the "log level: Any" filter is chosen.

The above log.info() statement shows up in Stackdriver as so:

[2m2019-10-01 17:55:41.159[0;39m [32m INFO[0;39m [35m1[0;39m [2m---[0;39m [2m[nio-8080-exec-5][0;39m [36mc.g.o.Application [0;39m [2m:[0;39m Hello world

with the Json payload:

jsonPayload: { container: {} instance: {} message: "[2m2019-10-01 17:55:41.159[0;39m [32m INFO[0;39m [35m1[0;39m [2m---[0;39m [2m[nio-8080-exec-5][0;39m [36mc.g.o.Application [0;39m [2m:[0;39m Hello world" }

and "logname" is projects/my-project/logs/gcplogs-docker-driver.

Why isn't Stackdriver capturing the log levels from Slf4j even though gcplogs-docker-driver is being used?


Solution

  • It looks like Docker's gcplogs-docker-driver causes the output to be sent to GCP's Stackdriver Logging (aka Cloud Logging). The gcplogs driver just sends each input line as-is with no further processing. There doesn't seem to be any appetite in docker/moby to do additional processing such as attempting to extract severities.

    You might be able to do some after-the-fact labelling, but I've never tried.

    Note that some platforms support performing additional processing before submitting the entries to Stackdriver Logging. For example, GKE logs console output using the Stackdriver Logging Agent which supports structured logs, which are JSON-encoded payloads. Or you might be able to configure your application's logging framework to log directly to Stackdriver Logging.