Using WildFly with embedded ActiveMQ Artemis as the JMS implementation. The queue is setup with a fixed memory size and FAIL action if queue is full. I need to be able to catch the resulting exception so the clients stop sending new messages. However, the exception doesn't seem to occur when calling send, but at the end of the transaction a few methods up. I'm not sure how to handle this properly.
If you're using a transaction then the messages are not actually sent when you call send on the JMS MessageProducer. They messages are only sent when the transaction is committed. Batching multiple operations together like this is one of the main features of transactions. If there is a problem sending the messages at the point of committing the transaction then the transaction commit will fail, but the specific MessageProducer which "sent" the messages will have no knowledge of it. You'll need to either stop using transactions or inform the sender of the problem another way (which will only be retroactive).
If you are only using transactions because you need to know the message reached the queue successfully and you don't need group multiple JMS operations into one logical unit then you can stop using transactions and either:
blockOnNonDurableSend=true in your client URL is you're sending non-durable (i.e. non-persistent) messages.In either case the client will wait for a response from the broker that the message was successfully received. You can read more about this in the ActiveMQ Artemis documentation.
If your client times out waiting for a response then that likely means the broker didn't receive the message and you are OK to send it again. However, in certain cases the broker (or the network) might fail after the message was successfully received but before the response could reach the client. In those situations if the client sends the message again it may result in a duplicate. To mitigate against the risk of duplicates you can set a duplicate ID on the message and use duplicate detection on the broker. You can read more about duplicate detection in the ActiveMQ Artemis documentation.
In any case, whether the messages are actually sent when send is invoked or later when commit is invoked the messages should not actually be sent if max-size-bytes has been reached since you're using FAIL.