I have a Cloud Scheduler job that invokes a Cloud Function every morning. The function simply creates a VM with a startup script that fetches data from a server. If there is new data that day, it downloads the data, pushes it to GCS and prints a message like "Fetching Successful". But if there is no new data, it simply prints a message like "No New Data". Then, the machine terminates.
I have been checking the result of the scheduled job everyday by manually going to Logging and checking the logs. If I see, "Fetching Successful", that's how I know the fetching that day was successful. If I see, "No New Data", then I know there was no new data that day.
But, I am growing really tired of scrolling through the thousands of logs everyday. Is there a way to set up an alert system, like when Logging detects "Fetching Successful" or "No New Data"?
I don't want a solution like setting a trigger to GCS, which will let me know when data has been uploaded... I want a solution within Logging if possible.. and preferably notifications.
Thank you in advance for your interest.
You can use logs-based metrics, as it enables viewing the incident in Cloud Monitoring charts and enables alerting policies. This document will help you to create alert policy.