Search code examples
pythonapscheduler

How to use evironment variables to set cron like schedule using apscheduler?


I have put together a ETL job written in Python. I have to run it through different environments (LOCAL, DEV, TST, PROD) so I need to find a way to set the job schedule using environment variables so that I can set different schedules for each environment without touching the code from one environment to another. Today I am using apscheduler. I have something like this in the code:

from apscheduler.schedulers.blocking import BlockingScheduler

def etl_job():


sched = BlockingScheduler(daemon=True)
sched.add_job(etl_job,'interval',minutes=7)
sched.start()

Does anyone know now to solve this? Thanks!


Solution

  • Here how I've solved this one:

    Environment variable (json format)

    schedule='{"jobsSchedule": [{"job_name": "job_1","schedule": "*/1 * * * *"},{"job_name":"job_2","schedule": "*/2 * * * *"}]}'
    

    Read the env variable

    jobCrontabSchedule = json.loads(os.getenv('schedule'))
    

    Schedule each one of the jobs

    for job in jobCrontabSchedule['jobsSchedule']:
        print ("Scheduling the job: "+job['job_name']+" with the schedule: "+job['schedule'])   
        sched.add_job(globals()[job['job_name']], CronTrigger.from_crontab(job['schedule']))