Scheduling a job after all jobs of a certain type are finished

Question

I have a rather complex caching and invalidation setup where I need to recalculate a lot of data if a specific dataset changes.

Basically if one specific entry changes, this might create up to 15 jobs recalculating stuff. While those jobs are running, it could happen that another of the main entries changes. Again creating several jobs. (This could happen simultaneous)

What I need to achieve is to aggregate the results after the jobs are run. And it would probably make sense to do this aggregation only once.

So what I need to do is run a single job, only once after all those other jobs are finished.

(BTW: I am unsing BCCResqueuBundle)

Answer 1

You could use a reference counter, and if needed combine it with a lock.

The reference counter tells the nr of async workers. Increment when started, decrement when finished or failed.

The lock: Either 1 or many async workers are running, or the aggregate job is running. If you use careful scheduling, you don't need a lock.