How to design a system with pub/sub with time-based coordination
I am working on a high-level design for a pub/sub model where the subscribers should not process messages until a specified time. For example, let's say a set of stock orders need to be cancelled exactly at some predefined expiration datetime. Let's say we have a message queue where each message has information on the order that needs to be cancelled. The queue is prioritized so that the messages with expiration time closest to the current time is at the head of the queue.
We obviously don't want the subscribers to work as fast as possible and cancel orders immediately. We want to only take the message from the queue if its expiration time is equal to or less than the current time.
Are there distributed message queues that support this use case? So that it blocks consumers from reading until a certain time? Or am I approaching this problem in an unreasonable way altogether?
AWS Step Functions could be used for this. You define a Wait state that receives an input parameter with a timestamp, and once that timestamp is hit, it will call a Task that you define.
For example:
I don't know how this works under the hood, but I'd imagine it puts messages in a main queue, and a poller continually reads messages off this queue. If currentTime < messageTimestamp then it puts it back, otherwise it puts to a different queue which is meant to be picked up by workers that actually do things with those messages.