Memory error when Loading large memory zip file Asp .Net C#

Question

I am using MemoryStream but it will error when the file has large memory and is Out of Memory. How can the controller download a large zip file >500mb? Because my file is already 800MB. Please help me with the solution in controller.

[HttpGet(ZipFileMinhChung)]
[ProducesResponseType(typeof(VoidMethodResult), (int)HttpStatusCode.BadRequest)]
public async Task<IActionResult> ZipFileMinhChungAsync([FromQuery] BaoCaoDanhMucMinhChungRequestViewModel request)
{
    var result = await _reportQueries.ExportZipFileMinhChungAsync(request).ConfigureAwait(false);

    var zipName = $"FilesMinhChung-{DateTime.Now.ToString("yyyy_MM_dd_HHmmss")}.zip";
    using (MemoryStream ms = new MemoryStream())
    {
        using (var archive = new ZipArchive(ms, ZipArchiveMode.Create, true))
        {
            foreach (var item in result)
            {
                using (HttpClient client = new HttpClient())
                {
                    var mediaUrl = _configuration["Gateway"] + item.FilePath;
                    // Download the file content from the external link
                    Stream fileContent = await client.GetStreamAsync(mediaUrl);

                    // Create a temporary file to store the file content
                    string tempFilePath = Path.GetTempFileName();

                    // Write the file content to the temporary file
                    using (FileStream fileStream = new FileStream(tempFilePath, FileMode.Create, FileAccess.Write))
                    {
                        await fileContent.CopyToAsync(fileStream);
                    }

                    // Copy stream file to zip
                    archive.CreateEntryFromFile(tempFilePath, "TieuChi" + item.MaTieuChuanTieuChi + "/" + item.MaMinhChung + "/" + item.FileName);
                }
            }
        }
        ms.Position = 0;
        return File(ms.ToArray(), "application/zip", zipName);
    }
}```

Answer 1

C# has an issue with byte arrays larger than 2Gb. Memory stream will use an array for storage, and it will over allocate to ensure it has enough space.

The absolutely simplest solution is to just specify the capacity when allocating the memory stream. Setting it to int.MaxValue should avoid problems for sizes smaller than 2Gb.

The next simplest solution will probably be to use a file-stream for the zip-archive:

// Create a temporary file to store the file content
string tempFilePath = Path.GetTempFileName();
using var fs = File.Create(tempFilePath);
using (var archive = new ZipArchive(fs, ZipArchiveMode.Create, true))
{
...
}
fs.Position = 0;
return File(fs, "application/zip", zipName);

Note that you can, and probably should, copy data directly to zip-archive entries:

var entry = archive.CreateEntry("TieuChi" + item.MaTieuChuanTieuChi + "/" + item.MaMinhChung + "/" + item.FileName);
using var entryStream = entry.Open();
await fileContent.CopyToAsync(entryStream );

Note that you probably want some way to cleanup temporary files, but I'm not sure what the recommended way to do that is. An alternative could be to create your own memory stream that uses multiple byte-arrays for data storage.