I have node js application where a user can schedule 4-5 different cron jobs. A user selects a start_date, end_date, start_time, and end_time, and accordingly, the cron functions are scheduled and some campaigns are generated to connect with leads (or other users). I am using npm package node-schedule to achieve the desired cron-job execution but the issue arises if my web application crashes in between.
As per the official documentation of node-schedule,
Note that Node Schedule is designed for in-process scheduling, i.e. scheduled jobs will only fire as long as your script is running, and the schedule will disappear when execution completes.
The solution I can come up to resolve this is to call the cron functions using server.js file which restarts every time the server script is executed. But I am unable to resolve some of the following issues:
Suppose, my scheduler is due to execute between 28th May 11:00 AM to 31st May 12:00 AM. The scheduler starts its job and some of the campaigns are executed in between. If my server goes down on 30th May, how can I pick up from the remaining date-time and campaigns and complete it from there?
So I am concerned about how to schedule or handle the cron job functions for the campaigns? If any of the campaign cron jobs is crashed in between if the server is stopped, how should I be able to re-schedule or execute that cron function from the time or date that has been left of it?
I am not much experienced with cron jobs, hoping for some handy solutions.
Here some advice, that you could review or implement depending of your hardware capabilities:
Decouple your current application in two separated artifacts: web and campaign. This allows each service to be deployed, rebuilt, re-deployed and managed independently. And also to have autonomous or cross-functional Teams.
source: https://skelia.com/articles/5-major-benefits-microservice-architecture/
If you need an app that doesn't affect a failure , crash or any event who needs a restart, you must use something outside to your app.
Commonly this outside is a database. You could use a json file if a database is not an option for you.
We have two entities here:
campaign_configurations
| identifier | desc | cron_expression |
|-------------------|----------------|-----------------|
| month_discounts | discounts ... | 0 0 1 * * |
| reminder_shop_car | a reminder ... | */10 * * * * |
| | | |
Also you can add a column to control the period validity of the cron with a start and end date.
campaign_executions
| execution_date | execution_status | execution_log |
|---------------------|------------------|---------------|
| 10-10-2020 01:00:00 | finished | ... |
| 10-10-2020 14:30:00 | error | ... |
| 10-10-2020 14:35:00 | executing | ... |
When your nodejs campaign starts/restarts due to failure or scheduled maintance, this must be the flow:
When app is healthy, its must query to the campaign_configurations table in order to execute new or modified cron expressions (previous explained flow)
This could be at every 30 minutes, more and less, according to your internal business logic.
You could use node-schedule for this task.