Search code examples
pythondjangoscheduled-tasks

How to use python to schedule tasks in a Django application

I'm new to Django and web frameworks in general. I have an app that is all set up and works perfectly fine on my localhost.

The program uses Twitter's API to gather a bunch of tweets and displays them to the user. The only problem is I need my python program that gets the tweets to be run in the background every-so-often.

This is where using the schedule module would make sense, but once I start the local server it never runs the schedule functions. I tried reading up on cronjobs and just can't seem to get it to work. How can I get Django to run a specific python file periodically?

Solution

  • I've encountered a similar situation and have had a lot of success with django-apscheduler. It is all self-contained - it runs with the Django server and jobs are tracked in the Django database, so you don't have to configure any external cron jobs or anything to call a script.

    Below is a basic way to get up and running quickly, but the links at the end of this post have far more documentation and details as well as more advanced options.

    Install with pip install django-apscheduler then add it to your INSTALLED_APPS:

    INSTALLED_APPS = [
    ...
    'django_apscheduler',
    ...
]

    Once installed, make sure to run makemigrations and migrate on the database.

    Create a scheduler python package (a folder in your app directory named scheduler with a blank __init__.py in it). Then, in there, create a file named scheduler.py, which should look something like this:

    from apscheduler.schedulers.background import BackgroundScheduler
from django_apscheduler.jobstores import DjangoJobStore, register_events
from django.utils import timezone
from django_apscheduler.models import DjangoJobExecution
import sys

# This is the function you want to schedule - add as many as you want and then register them in the start() function below
def deactivate_expired_accounts():
    today = timezone.now()
    ...
    # get accounts, expire them, etc.
    ...


def start():
    scheduler = BackgroundScheduler()
    scheduler.add_jobstore(DjangoJobStore(), "default")
    # run this job every 24 hours
    scheduler.add_job(deactivate_expired_accounts, 'interval', hours=24, name='clean_accounts', jobstore='default')
    register_events(scheduler)
    scheduler.start()
    print("Scheduler started...", file=sys.stdout)

    In your apps.py file (create it if it doesn't exist):

    from django.apps import AppConfig

    class AppNameConfig(AppConfig):
        name = 'your_app_name'
        def ready(self):
            from scheduler import scheduler
            scheduler.start()

    A word of caution: when using this with DEBUG = True in your settings.py file, run the development server with the --noreload flag set (i.e. python manage.py runserver localhost:8000 --noreload), otherwise the scheduled tasks will start and run twice.

    Also, django-apscheduler does not allow you to pass any parameters to the functions that are scheduled to be run. It is a limitation, but I've never had a problem with it. You can load them from some external source, like the Django database, if you really need to.

    You can use all the standard Django libraries, packages and functions inside the apscheduler tasks (functions). For example, to query models, call external APIs, parse responses/data, etc. etc. It's seamlessly integrated.

    Some additional links: