I have a .NET Core service. We want to implement fire and forget call for one of its endpoint. When request comes to this end point it should fire a method and immediately returns response as Ok. The method then processes that requests by calling other services, and some DB operations.
The above works fine. But, when I tried to do perf testing with thousands of requests, sent back to back, we noticed that some requests are not getting processed at all. It is working fine up to 4000 requests but more than that it is not processing some(approx 20+) requests. There are no exceptions in the logs
How can I identify the issue?
await Task.Factory.StartNew(() => FireAndForgetMethod());
return Ok();
"Fire and forget" literally means that you don't care if the completes and you're perfectly fine with ignoring exceptions. Since you do care that it completes and you do want to see exceptions, then you do not have a situation that is appropriate for "fire and forget".
When request comes to this end point it should fire a method and immediately returns response as Ok. The method then processes that requests by calling other services, and some DB operations.
Your API endpoint should serialize the work to be done into a message and place it on a reliable message queue (e.g., Azure Queue, Amazon Simple Queue, etc), after which it may return an HTTP result to its caller. This queue should then be read and each message executed by a background processor (e.g., ASP.NET Core Worker Service).