Spring boot: Download zipped large files while they are being processed on the server

Question

In my app I'm zipping and then downloading larges files, the files are located in azure, so I read the files from a stream and then zip them one after another, so I can dowload the zip file after all files has been zipped, here's my code:

@RequestMapping(value = "{analyseId}/download", method = RequestMethod.GET, produces = "application/zip")
public ResponseEntity<Resource> download(@PathVariable List<String> paths) throws IOException {

    String zipFileName = "zipFiles.zip";
    File zipFile = new File(zipFileName);
    FileOutputStream fos = new FileOutputStream(zipFile);
    ZipOutputStream zos = new ZipOutputStream(fos);
    for (String path : paths) {
        InputStream fis = azureDataLakeStoreService.readFile(path);
        addToZipFile(path , zos, fis);
    }
    zos.close();
    fos.close();
    BufferedInputStream zipFileInputStream = new BufferedInputStream(new FileInputStream(zipFile.getAbsolutePath()));
    InputStreamResource resource = new InputStreamResource(zipFileInputStream);
    zipFile.delete();

    return ResponseEntity.ok()
            .header(HttpHeaders.ACCESS_CONTROL_EXPOSE_HEADERS, HttpHeaders.CONTENT_DISPOSITION)
            .header(HttpHeaders.CONTENT_DISPOSITION, "attachment;filename=" + zipFileName)
            .contentType(MediaType.parseMediaType("application/octet-stream"))
            .body(resource);
}

private static void addToZipFile(String path, ZipOutputStream zos, InputStream fis) throws IOException {
    ZipEntry zipEntry = new ZipEntry(FilenameUtils.getName(path));
    zos.putNextEntry(zipEntry);
    byte[] bytes = new byte[1024];
    int length;
    while ((length = fis.read(bytes)) >= 0) {
        zos.write(bytes, 0, length);
    }
    zos.closeEntry();
    fis.close();
}

However on azure the request time out is set to 230 sec, and cannot be changed, however for big files it takes more than that to load and then zip the files on the server, so the connection with the client will be lost meanwhile.

So my question is since I'm getting the data from a stream, can we do all these operations simultaneously, means getting the stream and download it as the same time and not waiting till getting the whole file, or if there any other idea can any body share it here please.

Thanks.

Answer 1

The answer is to not download the file to the server and then send it to the client but streaming it to the client directly here's the code

@RequestMapping(value = "/download", method = RequestMethod.GET)
public StreamingResponseBody download(@PathVariable String path) throws IOException {

    final InputStream fecFile = azureDataLakeStoreService.readFile(path);
    return (os) -> {
        readAndWrite(fecFile, os);
    };
}

private void readAndWrite(final InputStream is, OutputStream os)
        throws IOException {
    byte[] data = new byte[2048];
    int read = 0;
    while ((read = is.read(data)) >= 0) {
        os.write(data, 0, read);
    }
    os.flush();
}

I also added this configuration to ApplicationInit:

@Configuration
public static class WebConfig extends WebMvcConfigurerAdapter {

    @Override
    public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
        configurer.setDefaultTimeout(-1);
        configurer.setTaskExecutor(asyncTaskExecutor());
    }

    @Bean
    public AsyncTaskExecutor asyncTaskExecutor() {
        return new SimpleAsyncTaskExecutor("async");
    }

}