We have a REST API built with Spring Boot, JPA and Hibernate. The clients using the API has an unreliable access to network. To avoid having too many errors for the end user, we made the client retry unsuccessful requests (eg. after a timeout occurs).
As we cannot be sure that the request has not already been processed by the server when sending it again, we need to make the POST requests idempotent. That is, sending twice the same POST request must not create the same resource twice.
To achieve this, here is what I did:
So far so good.
I have multiple instances of the server working on the same database, and requests are load balanced. As a result, any instance can process the requests.
With my current implementation, the following scenario can occur:
In this scenario, the request has been processed twice, which is what I want to avoid.
I thought of two possible solutions:
I'm using a Filter to retrieve and store a response. My filter looks roughly like this:
@Component
public class IdempotentRequestFilter implements Filter {
@Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
String requestId = getRequestId(request);
if(requestId != null) {
ResponseCache existingResponse = getExistingResponse(requestId);
if(existingResponse != null) {
serveExistingResponse(response, existingResponse);
}
else {
filterChain.doFilter(request, response);
try {
saveResponse(requestId, response);
serve(response);
}
catch (DataIntegrityViolationException e) {
// Here perform rollback somehow
existingResponse = getExistingResponse(requestId);
serveExistingResponse(response, existingResponse);
}
}
}
else {
filterChain.doFilter(request, response);
}
}
...
My requests are then processed like this:
@Controller
public class UserController {
@Autowired
UserManager userManager;
@RequestMapping(value = "/user", method = RequestMethod.POST)
@ResponseBody
public User createUser(@RequestBody User newUser) {
return userManager.create(newUser);
}
}
@Component
@Lazy
public class UserManager {
@Transactional("transactionManager")
public User create(User user) {
userRepository.save(user);
return user;
}
}
Filter shown above? Is it a good practice? @Transactional("transactionManager"). What will happen when I start or rollback a transaction with the filter?Note: I am rather new to spring, hibernate and JPA, and I have a limited understanding of the mechanism behind transactions and filters.
Based on
To avoid having too many errors for the end user, we made the client retry unsuccessful requests
you seem to have full control of the client code (great!) as well as the server.
It is, however, not clear whether the problem with the client's network is flakiness (the connection often randomly drops and requests are aborted) or slowness (timeouts), since you've mentioned both. So let's analyse both!
The first things that I'd recommend are:
However:
then the standard request-response scheme is probably not suitable.
In this case, instead of having the client wait for a response you could perhaps send back an immediate acknowledgement Request received and send the actual response via some TCP socket? Any following attempts would receive either a message saying that the Request is being processed, or the final response, if the operation is complete (this is where the idempotence of your operation would help).
If the client network is flaky and prone to frequent failures, the above proposed solution, where requests and responses are uncoupled, should work too!
Request in progress, you could try listening on the socket again.If using a socket is not an option, you could use polling. Polling isn't great but personally, I'd most likely still go with polling rather than rollbacks, especially if the server operations are slow - this would allow for decent pauses before retries.
The problem with rollbacks is that they'd try to recover from failures using code, which in itself is never foolproof. What if something goes wrong while rolling back? Can you make sure that the rollback is atomic and idempotent, and will never, under any circumstances, leave the system in an undefined state? That's beside the fact that they can be non-trivial to implement and would introduce additional complexity and extra code for testing and maintenance.
You'll have more trouble if you don't own the client code, as the consumer of your API would be free to make lots of arbitrary calls to your servers. In this case I would definitely lock idempotent operations and return responses saying that the request is being processed instead of trying to revert anything using rollbacks. Imagine having multiple concurrent requests and rollbacks! If you were not happy with Stanislav's proposal (The queue will get longer and longer, making the whole system slower, reducing the capacity of the system to serve requests.), I believe that this scenario would be even worse.