I have an application which consumes messages from a queue on my AMQ broker. I may need multiple instances of this application (for performance reasons), and in this case each instance should consume a subset of the messages coming from the queue. This subset should be chosen based on the value of 'HeaderA' header on the incoming message.
I.e.
(Note: I know this design isn't great and it would be better if all instances where equivalent and could consume any message, but I am constrained to this design for external reasons)
I am considering two approaches;
Route messages on the Broker using Apache Camel so that there is a separate queue (created dynamically) for each unique value of 'HeaderA'. Each instance then dynamically subscribes to the queue it is interested in.
Each instance creates a consumer on the same queue but uses a JMS Selector to only consume those messages it is interested in.
Which approach would be considered 'best' and why? Is creating potentially thousands of individual queues for unique header values dangerous? Are there any caveats to using JMS Selectors?
I would go for selectors.
Even though you could, theoretically, create queues dynamically on the fly most (all?) JMS providers such as ActiveMQ are not optimized to scale with thousands of queues. I should point out, it may work though. Just checkout that your setup does not grow in thread usage or file descriptor usage (or similar resources).
There is no big pain using JMS selectors. One potential problem might be that a type-o in the selector may consume too many messages or no messages at all without indicating errors. Extensive system testing is needed, as always.
Another thing - the management tools such as ActiveMQ Admin console and Hawt.io is not built to support a large number of queues. If you intend to use them, consider not using more queues than needed.