We have an akka/scala app that has some naively written error handling that I need to fix.
The REST endpoint communicates with an internal actor that makes a remote call to create an order.
It does this using ask, and when the ask times out e.g. because of a network or comms error, we send the client a message over the REST endpoint that the request has failed.
The problem is that the internal actor has its own queuing/retry logic and it will continue to call the remote interface until the request succeeds.
So we have the situation where we've told the client that the request has failed but it's really just queued (and will often eventually succeed). The client resubmits the request and we end up with 100's of duplicate orders.
My question is: does akka support a generic way of rolling back or poisoning an ask message when the ask request times out?
There are a couple of approaches one can take. Neither is generic, as (especially once network communication is involved) any choice made in this area has some cases where it's exactly the wrong thing for the business logic:
If there's something about the orders that can be used to determine that two submitted orders are actually the same (e.g. a client-supplied correlation ID), that can be used in the actor to piggyback on the queuing/retry logic handling the earlier order. This requires some client-visible API changes.
It's also possible to include a stop retrying and ignore if pulled from a queue after this time field in the message; you can set this time based on the ask timeout duration.