I need to implement a system for running asynchronous operations in server side, and after pondering many options I am not sure which would be the best and more scalable, or what is normally used in this scenarios. I must be missing something since none of the solutions completely solve the problem.
Context: For some context, I have a mobile application where users send a request to other users in order to start an activity. This operations are handled server side, and if the request doesn't have a response from the other user, it will expire and be closed in 15 minutes. So I create a record with the starting time of this operation in the database and I need to update its status 15 minutes later, hence the asynchronous operation. It needs to be exactly 15 minutes in order to notify the waiting user properly.
Possible Solutions:
I am using laravel, and the solution I am pondering is pushing delayed jobs to a queue. Although at the beginning I thought it would be the perfect solution, I think it may not be scalable since the frequency of this requests can become too high and too many jobs would be queued. Also, queue jobs are not guaranteed to be executed exactly after the delay is reached, and may be resolved too late.
Another solution I thought about was handling all of this in a cron job running every minute, but it also has the same problem when the volume of requests is high. Performance-wise it may be overkill to do this every minute, especially at times where activity is infrequent.
So basically, these are small operations which would be relatively fast to resolve, but the cost of managing a high amount of pending operations may be worst than the operations themselves. Is there any other pattern (not even reduced to laravel nor php) to delay the execution of a task to a future time?
I think you want the determination of user requests needing action to be done by the database/application (i.e. scope by an expires column), not by the queue daemon. But you want that determination to happen at a dynamic rate.
One possible solution in Laravel:
Create a meta-job that is added to the queue on startup and re-queues itself at a dynamic rate. In other words, you have only one job on your queue, and it evaluates all pending user requests. When it's done, it releases itself back into the queue, with a delay that depends on the number of still-pending requests (more pending requests = shorter delay, less pending requests = longer delay). The user requests are never put into the queue themselves, just processed by this dynamically-delayed meta-job.
Also, just as an FYI, the Laravel queue:work --daemon uses sleep() to delay its queue-checking loops, which is probably more efficient than using a cron job (or a queue:listen process), which reloads the entire app every time it's called, but at the trade-off of having to be manually re-started to pull in any changes you may have made to the app.