Search code examples
google-bigquerydata-transfer

Add alert when transfer in BigQuery results in "succeeded 0 jobs"


I have a schedule transfer running daily on BigQuery and mostly without any issues. The transfer reads a .csv file from an AWS S3 bucket and appends the information to a BigQuery table.

Recently there has been an issue where the transfer resulted in neither succeeded nor failed jobs.

transfer logs

The outcome was that no entries were imported but also no alert was triggered; I had to hear from the reports' users that something had gone wrong.

Question: is there a way to add an alert on BigQuery Transfers for when successful jobs = 0?


Solution

  • BigQuery does have monitoring, it has some known issues as well, this will help BigQuery Monitoring

    Monitoring->Dashboard->Add Chart->Use Resource type Global and Metric type as "Uploaded rows".