ASP.NET MVC: Uploading files in chunks with Azure Blob Storage

Question

Is there any way to upload a large files in chunks in ASP.NET MVC? I am trying to upload a file, 5 MB, as a test and I want to split it into 1 MB chunks and upload each chunk individually. I am doing this test before trying it with files that are greater than 1 GB to see whether it is possible on my current application.

This is my current code for uploading files within blobs without splitting them (Assuming that the container exists). This code is tested and working.

// connectionString is self explanatory
// files = array of files to be uploaded which contains the file count, names, streams
// blobContainer = self explanatory
public async Task<string> UploadFiles(connectionString, files, blobContainer) {
  try {
    BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
    BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(blobContainer);

    for (int i = 0; i < files.count; i++) {
      BlobClient blobClient = containerClient.GetBlobClient(file[i].FileName);
      await blobClient.UploadAsync(file[i].InputStream);
    }
  }
  catch (Exception e) {
    throw e;
  }
  return "Success";
}

I know the solution to my problem is to introduce blocks to my code to store the chunks of my files and AzureBlobStorage should understand that it should recombine those chunks back into the whole file. However, all I am trying and researching online as turned up short.

I searched online and found this reference (https://www.andrewhoefling.com/Blog/Post/uploading-large-files-to-azure-blob-storage-in-c-sharp) as the only guide that is the most similar to mine on how I could find on how to split and upload my files in chunks. However, you can see that the author uses CloudBlockBlob as the instance and not BlobClient like my application does and for some reason I cannot implement CloudBlockBlob, even when I am importing Azure.Blob.Storage. I am very confused on how to proceed from here

Answer 1

I tried the below ASP.NET MVC code and uploaded 5mb file to the Azure storage.

Code :

Controllers/HomeControler.cs :

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Specialized;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;

namespace LargeFileUpload.Controllers
{
    public class HomeController : Controller
    {
        private const string connectionString = "<connec_string>";
        private const string blobContainerName = "<container_name>";
        private const int chunkSize = 1024 * 1024; 

        [HttpGet]
        public IActionResult Index()
        {
            return View();
        }

        [HttpPost]
        public async Task<IActionResult> Upload(IFormFile file)
        {
            try
            {
                if (file == null || file.Length == 0)
                {
                    return RedirectToAction("Error");
                }

                BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
                BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(blobContainerName);
                string blobName = Guid.NewGuid().ToString();
                BlockBlobClient blockBlobClient = containerClient.GetBlockBlobClient(blobName);

                using (Stream stream = file.OpenReadStream())
                {
                    byte[] buffer = new byte[chunkSize];
                    int bytesRead;
                    int blockNumber = 0;

                    while ((bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length)) > 0)
                    {
                        using (MemoryStream chunkStream = new MemoryStream(buffer, 0, bytesRead))
                        {
                            string blockId = Convert.ToBase64String(BitConverter.GetBytes(blockNumber));
                            await blockBlobClient.StageBlockAsync(blockId, chunkStream);

                            blockNumber++;
                        }
                    }
                    var blockList = Enumerable.Range(0, blockNumber)
                        .Select(n => Convert.ToBase64String(BitConverter.GetBytes(n)))
                        .ToList();
                    await blockBlobClient.CommitBlockListAsync(blockList);
                }

                return RedirectToAction("Success");
            }
            catch (Exception ex)
            {
                return RedirectToAction("Error");
            }
        }

        public IActionResult Success()
        {
            return View();
        }

        public IActionResult Error()
        {
            return View();
        }
    }
}

Views/Home/Error.cshtml :

@{
    ViewData["Title"] = "Error";
}

<h2>Error</h2>
<p>An error occurred during the file upload process. Please try again later.</p>

Views/Home/Index.cshtml :

@{
    ViewData["Title"] = "File Upload";
}

<h2>Upload a File</h2>

<form asp-action="Upload" method="post" enctype="multipart/form-data">
    <div class="form-group">
        <label for="file">Select File:</label>
        <input type="file" name="file" id="file" class="form-control" required>
    </div>
    <button type="submit" class="btn btn-primary">Upload</button>
</form>

Views/Home/Success.cshtml :

@{
    ViewData["Title"] = "Upload Success";
}

<h2>Upload Successful</h2>
<p>Your file has been successfully uploaded to Azure Blob Storage.</p>

Below is my project structure:

Output :

It runs successfully as below:

Then, the below page opened automatically in the browser.

I clicked on Choose File and selected a 5mb file. Then, I click on Upload button as below.

Below is my 5mb file data that I want to upload.

The blob uploaded successfully, as shown below in the browser.

The blob uploaded successfully to my storage account in Azure Portal, and then I clicked on download to see the data inside my blob as below,

Below is the downloaded blob data,