Logging using Stackdriver API on Kubernetes / Google Container Engine (GKE)

Question

I have a go application that leverages Google Cloud Logging API.

The relevant code is the same as this sample from their documentation: https://github.com/GoogleCloudPlatform/golang-samples/blob/master/logging/logging_quickstart/main.go

After checking it works with minikube (my payload appears in the Global category of my logs viewer), I deploy the app on Google Container Engine (GKE).

Once deployed there, I can no longer see the logs the app sends through the logging API. The logs written to std appears in the GKE container category, but no trace of the entries I send using the API.

My cluster has Stackdriver logging API enabled / write only, the default service account is Editor (even tried with Owner), I also tried with a dedicated service account (using the env GOOGLE_APPLICATION_CREDENTIALS) with log writer or even owner access, I can’t see the logs and no error are reported from the client library.

What could be the cause or where could I start debugging such issue?

Thank you,

Answer 1

So turns out the log were there but not where I'd expect them.

Using the gcloud CLI I could see those logs got the resource type gce_instance and therefore appears in the GCE VM Instance category

To figure this out:

  $ gcloud beta logging logs list
  projects/<PROJECT>/logs/<LOG_NAME>
  ...

then

$ gcloud beta logging read projects/<PROJECT>/logs/<LOG_NAME>
---
insertId: ...
jsonPayload:
   ...
logName: ...
receiveTimestamp: ...
resource:
  labels:
    instance_id: ...
    project_id: ...
    zone: ...
  type: gce_instance
timestamp: ...

Note type being gce_instance