Search code examples
google-app-enginegoogle-cloud-platformgcloudgoogle-cloud-stackdriver

Logs are not getting exported at log sink as per the inclusion filter of timestamp

I wanted to export the logs of my AppEngine service to the cloud storage bucket. Hence, I created the log sink & I gave the destination of my cloud storage bucket.The problem is that logs are not getting exported as per my inclusion filter condition. Inclusion Filter:-

resource.type="gae_app"
resource.labels.module_id="cbd-iris-app"
timestamp>="2021-07-17T01:20:00Z" AND timestamp<="2021-07-17T16:30:00Z"

As per the filter, I am expecting the logs to be between the timestamps mentioned in code. But, instead the logs which are getting stored are of current date & time. What exactly is wrong here ..

Solution

  • TL;DR: This is the expected behavior for GCP's Log routing

    Based on How sinks work section, looks like you can only apply to new logs and not to logs that had been already processed:

    Caution: Since routing happens for new log entries only, you can't route log entries that Logging received before your sink was created.

    So although you set a timestamp filter, the sink can only take logs that are created after the sink was created.