Search code examples
pythondistributed-systemapscheduler

APScheduler. What is the best practices to shutdown schedulers with any non memory storage?

I need to run scheduler distributed to many machines and has one single point to store jobs. I use redis for this purposes and my code looks like this:

jobstores = {'default': {'type': 'redis'}}
scheduler = BlockingScheduler(jobstores=jobstores)
scheduler.add_job(...)

When I place jobs into scheduler and call .start() it add jobs in redis. But when I run another instance of my scheduler or when I stop current one and re-run it again, scheduler add more jobs in redis (doesn't do cleanup).

Therefore I have this question. What is the best practice to do such things?

Thanks in advance.

Solution

  • You must use the replace_existing=True option while you are adding a new job for this kind of usage.

    From the documentation:

    If you schedule jobs in a persistent job store during your application’s initialization, you MUST define an explicit ID for the job and use replace_existing=True or you will get a new copy of the job every time your application restarts!