Search code examples
pythondjangolinuxcronscheduled-tasks

Best practice for creating and maintaining multiple Python scripts on a server


I have a Ubuntu server with multiple Python scripts. They all run at different times of the days and access our Django database to store values they retrieve from other processes(file scraper for example to save row entries in our DB). Right now we run all our scripts using cron but I wanted to know what the best practice of method of implementing this would be?

I was hoping on being able to set it up as a service and have it send out emails if any of the scripts crash and along with that, if they do then auto restart them.

Is cron the best way or are there better methods?


Solution

  • Two popular Python schedulers are airflow and luigi. Lots of articles online about both.

    Luigi is especially popular for people running windows (since airflow only runs on Linux and MacOS). But since you are running Ubuntu, both options should work for you.