I am scheduling a task in Cloud Run which injects data from a Firestore export into BiqQuery using the bq
command line tool. This takes a while.
I discovered that the Cloud Scheduler Attempt Deadline doesn't match Cloud Run Maximum tasks timeout. It is 30mins vs 60mins.
I don't need to display a failure in Cloud Scheduler as I can use regular error monitoring, so I plan to respond to the scheduler request before the cloud run process has finished.
Is it ok to do this, or will the Cloud Run process potentially be killed by the auto-scaling mechanism after the HTTP response has been sent?
You can read the life cycle container here in the documentation. In summary, by default the CPU(s) is only allowed to the Cloud Run service during the request processing, else it is throttled.
You can set the throttling to false to let the CPU ON on the Cloud Run instance after the response has been sent (and you will pay accordingly). If no other request is received on the instance, the autoscaler will kick it after 15 minutes (idle activities).
To prevent that, you can set a min instance > 0, like that, a minimal number of instance are kept up and running every time (and you will also pay for it).
Or..... Cloud Run is not the right service for you, maybe Batch?