Search code examples
pythongoogle-cloud-platformstackdrivergoogle-cloud-stackdriver

How to filter GCP stackdriver logs by timestamp from python

I'm trying to get GCP stackdriver logs from python module. I can filter logs from request_id and user

but when I'm trying to get logs between two time periods it gives my error.

import json
import sys
from google.cloud import logging
from google.cloud.logging import DESCENDING
from google.cloud import bigquery

#stackdriver client object
logging_client = logging.Client()

FILTER="logName:projects/cognitedata-europe-west1-1/logs/stdout AND timestamp>="+sys.argv[3]
results_iterator = logging_client.list_entries(filter_=FILTER,projects=PROJECT_IDS,order_by=DESCENDING,page_size=1000)

I use following pattens to provide time.

timestamp:sys.argv[3]

timestamp=sys.argv[3]

timestamp>sys.argv[3]

timestamp>=sys.argv[3]

But noting solves my issue.

I have input time for sys.argv3 as "2019-09-28T03:28:04.189Z"

My requirement is to filter data from the timestamp enter image description here

But in GCP stack driver logger UI it gives results for both 

timestamp:"2019-09-28T03:28:04.189Z"

enter image description here

and 

timestamp>="2019-09-28T03:28:04.189Z" AND timestamp<="2019-09-28T03:30:39.189Z"

enter image description here

Solution

  • As mentioned by DazWilkin in the comments, you need to include the double quotes in the filter as well. You can do something like:

    FILTER="logName:projects/cognitedata-europe-west1-1/logs/stdout AND timestamp>=" + "\"" + sys.argv[3] + "\""

    If the above looks too complicated to read, you can use the combination of Python's single and double quoted strings which would look something like below:

    FILTER='logName:projects/cognitedata-europe-west1-1/logs/stdout AND timestamp>=' + '\"' + sys.argv[3] + '\"'

    NOTE: Backward slashes(\) are being used to escape the double quotes(").