Search code examples
node.jsgoogle-cloud-platformcloudscheduled-tasks

What's the best solution to execute scheduled tasks for later?


This is not really a code related question, but more of a solutions oriented question.

To provide some context, I have a web application with a serverless Node.js backend which lets users define a recurring time period at which an endpoint will execute and perform some action. For example, a user decides that every 9 days, the action will be fired.

I am well aware of solutions such as GCP Cloud Tasks (which is what I'm currently using) and AWS SQS, but they have limits. For instance, GCP's Cloud Tasks has a max schedule limit of 720 hours (30 days). This means that my users can only schedule tasks at a future date within 720 hours in the future whereas I would prefer to give them the flexibility to schedule tasks for up to one year.

Is there currently a cloud solution that would allow me to perform such a feature?

I am suspecting that this is definitely possible because of Stripe's subscriptions. They allow yearly subscriptions, which are similar to what I'm using, but with an extended limit. I'm not exactly sure how Stripe engineers accomplished this under the hood (while keeping scalability as a requirement since cronjobs are quite expensive), but after googling solutions to no avail, this problem had me wondering if there existed a more appropriate solution or whether my solution is fine as it is.

I'm also aware that there are workarounds to "extend" the Cloud Task 720 hours limit, but I want to explore options before diving into those kind of workarounds. (e.g. Cronjobs for each user would work well but expensive at scale, One cronjob checking for all the current day's tasks to schedule but at scale, the cloud function might time out since I'm using serverless backend, etc.)


Solution

  • You should have a look to cloud Workflow. For each user request, launch an execution with the parameters, they can be "delay" or "url to call"; define those that you need.

    So, when an execution start, wait the delay, and then call the URL. An execution can last up to 1 year, that fit your requirements