Search code examples
pythonweb-servicesschedulinglong-running-processes

python long running daemon job processor

I want to write a long running process (linux daemon) that serves two purposes:

  • responds to REST web requests
  • executes jobs which can be scheduled

I originally had it working as a simple program that would run through runs and do the updates which I then cron’d, but now I have the added REST requirement, and would also like to change the frequency of some jobs, but not others (let’s say all jobs have different frequencies).

I have 0 experience writing long running processes, especially ones that do things on their own, rather than responding to requests.

My basic plan is to run the REST part in a separate thread/process, and figured I’d run the jobs part separately.

I’m wondering if there exists any patterns, specifically python, (I’ve looked and haven’t really found any examples of what I want to do) or if anyone has any suggestions on where to begin with transitioning my project to meet these new requirements. I’ve seen a few projects that touch on scheduling, but I’m really looking for real world user experience / suggestions here. What works / doesn’t work for you?

Solution

    • If the REST server and the scheduled jobs have nothing in common, do two separate implementations, the REST server and the jobs stuff, and run them as separate processes.

    • As mentioned previously, look into existing schedulers for the jobs stuff. I don't know if Twisted would be an alternative, but you might want to check this platform.

    • If, OTOH, the REST interface invokes the same functionality as the scheduled jobs do, you should try to look at them as two interfaces to the same functionality, e.g. like this:

      • Write the actual jobs as programs the REST server can fork and run.
      • Have a separate scheduler that handles the timing of the jobs.
      • If a job is due to run, let the scheduler issue a corresponding REST request to the local server. This way the scheduler only handles job descriptions, but has no own knowledge how they are implemented.

    • It's a common trait for long-running, high-availability processes to have an additional "supervisor" process that just checks the necessary demons are up and running, and restarts them as necessary.