How to handle background jobs in "Cloud Run" when new instance is created immediately?

Question

I have a FastAPI project in Cloud Run and it has some background jobs inside it. (Not heavy stuff)

However, when a new instance is being created by Cloud Run due to number of requests etc. every instance runs the background job concurrently.

For example;

I have a task that creates some invoices for customers in the background and if three instances is created immediately, three invoices will be created.

I researched about "FOR UPDATE" usage in PostgreSQL etc. It seems like I can solve by modifying my database but I just wonder if it can be solved in Cloud's side.

• I don't want to limit the max. number of instances to 1

What would you do in this situation?

Thank you for your time.

Answer 1

If you can potentially have N instances of a job (because you don't want to set the max limit to 1), you need to implement your jobs in an idempotent way. Broadly speaking, you have a few ways to achieve idempotency:

1. by enforcing a business constraint.
2. by storing an idempotency key.
3. by using the Etag HTTP response header.

For example, Stripe lets you define an idempotency key for all of your API requests. Stripe stores this key on its servers, and when you make a POST request with the same payload of a previous one, Stripe returns you the same result. POST requests are not idempotent, but using this "trick" they become idempotent.

Stripe's idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeded or failed. Subsequent requests with the same key return the same result, including 500 errors.

https://stripe.com/docs/api/idempotent_requests

Tip: you could expand your question by clarifying how these background tasks are created, and where they run.