My production servers has hundreds of users per block and I've realized that exporting data could potentially blow memory and ruin the app for multiple users.
We are talking about millions of data being exported by a single user.
Is there a way to create a CSV file and stream it to the front end as it is being generated to use as little memory as possible?
Making the front end request batches and generate the CSV file in the front end is not an option, this call will be used for other platforms and I'm trying to make it as clean as possible for all.
If you look at the Spring Framework Documentation on Spring Web MVC, section 1.4.3. Handler Methods, sub-section Return Values, you will find many ways to return streaming data, e.g.
void - A method with a void return type (or null return value) is considered to have fully handled the response if it also has a ServletResponse, or an OutputStream argument, or an @ResponseStatus annotation.
ResponseBodyEmitter - Emit a stream of objects asynchronously to be written to the response with HttpMessageConverter's; also supported as the body of a ResponseEntity. See Async Requests and HTTP Streaming.
That means you can do it:
Synchronous: Write the raw response yourself to the HTTP response stream in the handler method. Response is complete when method returns.
Asynchronous: Prepare the streaming (including HTTP headers) in your handler method, then do the actual streaming in another thread.