Search code examples
azureazure-web-app-service

Problem with network bandwidth of Azure App Service (very slow uploads)

My App Service (A) sends a POST request that contains a base64 encoded file to App Service B . I understand that this may seem strange, but it is needed in this case. App Service B uploads it to a Blob Storage and then returns a response. The size of the file is between 40-80mb.

The problem is that the file upload is really slow (sometimes longer than two minutes). I am trying to understand why it is so slow. I would assume that the network bandwith is quite high (the two app services are in the same region). Both app services use the S1 tier.

Here is the simplified code:

App service A Controller:

private static async Task UploadFileToAppServiceB(HttpClient client, string fileName, byte[] fileContent)
{
    var path = $"www.foo.bar/upload";
    var request = new
    {
        FileName = fileName,
        Content = fileContent,
    };
    client.Timeout = TimeSpan.FromMinutes(2);
    var response = await HandleNotFound(client.PostAsJsonAsync(path, request));
    response.EnsureSuccessStatusCode();
}

App Service B controller:

[HttpPost("upload")]
[RequestSizeLimit(209715200)]
public async Task UploadFileToBlobStorage([FromBody] FooFile fooFile)
{
    var fileContent = fooFile.Content;
    var blobContainerClient = GetContainerClient();
    var blobClient = blobContainerClient.GetBlobClient();

    await using var memoryStream = new MemoryStream(fileContent);
    await blobClient.UploadAsync(memoryStream, true);
}

Why does App Service B take so long to respond? Is there a way I can improve this?

Solution

  • To address the issue of slow uploads between Azure App Services and to Azure Blob Storage, you can follow these steps to optimize the performance.

    Upgrade App Service Plan so that can provide additional resources that improve network throughput and overall performance.

    1. Navigate to the Azure Portal.
    2. Go to your App Service.
    3. Click on "Scale up (App Service plan)" under Settings.
    4. Choose a higher-tier plan such as P1V2 or P2V3 and review the additional resources and features they offer.
    5. Click on "Apply" to upgrade your plan.

    enter image description here

    Modify Application to Handle Binary Data. Changing the application to handle binary data directly instead of using base64 will reduce the data size and improve transmission speed.

    *App Service A Code Modification: * Replace the base64 encoding mechanism with a binary data handling approach in your HTTP client request.

    private static async Task UploadFileToAppServiceB(HttpClient client, string fileName, byte[] fileContent)
{
    var path = $"https://foo.bar/upload";
    using var content = new ByteArrayContent(fileContent);
    content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
    var formData = new MultipartFormDataContent
    {
        { new StringContent(fileName), "FileName" },
        { content, "file", fileName }
    };

    var response = await client.PostAsync(path, formData);
    response.EnsureSuccessStatusCode();
}

    App Service B Code Modification: Change the controller to accept a file stream directly.

    [HttpPost("upload")]
[RequestSizeLimit(209715200)]
public async Task<IActionResult> UploadFileToBlobStorage(IFormFile file)
{
    var blobContainerClient = new BlobServiceClient(connectionString).GetBlobContainerClient("mycontainer");
    var blobClient = blobContainerClient.GetBlobClient(file.FileName);

    await using (var stream = file.OpenReadStream())
    {
        await blobClient.UploadAsync(stream, true);
    }
    return Ok();
}

    Enable Asynchronous Processing i.e. ensure the existing code above is using async-await properly to not block on I/O operations.

    Optimize Azure Blob Storage Configuration

    Switch to a higher-performance tier for Azure Blob Storage if you are using standard storage.

    1. Go to your Storage Account in the Azure Portal.
    2. Navigate to "Configuration" under Settings.
    3. Change the performance tier to Premium

    enter image description here

    Implement Application Insights for Monitoring

    Set up Application Insights for detailed performance monitoring and to identify any other bottlenecks.

    1. Go to the Azure Portal.
    2. Search for Application Insights and create a new resource or attach to an existing App Service.
    3. Follow the instructions to integrate Application Insights with your application.

    enter image description here

    Add Application Insights SDK to your projects:

    dotnet add package Microsoft.ApplicationInsights.AspNetCore

    Configure in Startup.cs:

    public void ConfigureServices(IServiceCollection services)
{
    services.AddApplicationInsightsTelemetry();
    // other services
}

    If the end-to-end latency metric for the Blob Storage is below 1 second, the focus should shift to the network and processing efficiency between App Service A and App Service B. So if you want to check network latency use tools like application insight as suggested above enter image description here

    then suppose if both services are within the same region but not in the same virtual network, consider configuring VNet Integration to potentially reduce latency and enhance security.

    These steps will provide a systematic approach to troubleshoot your slow file uploads better.

    References: