Search code examples
c#file-uploadlarge-filesmultifile-uploaderiformfile

c# net7 IFormFile: upload large files


I want to receive files in different sizes. I already get the problem with a 30 MB file.

If I use Postman as a client then I get problems with a 7 MB file when I, in intervals of 2 seconds, send the same request again: enter image description here

System.IO.InvalidDataException: Multipart body length limit 16384 exceeded.
   at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.UpdatePosition(Int32 read)
   at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.MultipartReader.ReadNextSectionAsync(CancellationToken cancellationToken)
   at FileTest.Services.Api.UploadController.Post() in ...\FileTest.Services\Api\UploadController.cs:line 37
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

HEADERS
=======
Accept: */*
Connection: keep-alive
Host: localhost:58638
User-Agent: PostmanRuntime/7.32.2
Accept-Encoding: gzip, deflate, br
Content-Type: multipart/form-data; boundary=--------------------------955915974361275464014603
Referer: http://localhost:58639/api/Upload
Content-Length: 7777366
Postman-Token: c580056c-7d65-4c1f-a497-f51a041b4d76

When I use a C# client, it just hangs. No reaction on server side.

Have now read one here on StackOverflow, Microsoft and others. But unfortunately always without success.

My goal is to transfer the following parameters in a request:

  • Id: string
  • Document: IFormFile
  • Attechments: IList

File sizes can finally be KB large but also GB large.

My code currently looks like this:

Program.cs

builder.Services.Configure<FormOptions>(options =>
{
    options.MultipartBodyLengthLimit = 50 * 1024 * 1024;
    options.BufferBodyLengthLimit = 50 * 1024 * 1024;
});

Controller

[Route("api/[controller]")]
[ApiController]
public class UploadController : ControllerBase
{
    [HttpPost]
    [DisableFormValueModelBinding]
    [RequestSizeLimit(50 * 1024 * 1024)]
    public async Task<IActionResult> Post()
    {
        var boundary = MultipartRequestHelper.GetBoundary(MediaTypeHeaderValue.Parse(Request.ContentType), int.MaxValue);
        var reader = new MultipartReader(boundary, HttpContext.Request.Body);

        var section = await reader.ReadNextSectionAsync();

        while (section != null)
        {
            section = await reader.ReadNextSectionAsync();
        }

        return Ok();
    }
}

Attribute DisableFormValueModelBinding

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class DisableFormValueModelBindingAttribute : Attribute, IResourceFilter
{
    public void OnResourceExecuting(ResourceExecutingContext context)
    {
        var factories = context.ValueProviderFactories;
        factories.RemoveType<FormValueProviderFactory>();
        factories.RemoveType<FormFileValueProviderFactory>();
        factories.RemoveType<JQueryFormValueProviderFactory>();
    }

    public void OnResourceExecuted(ResourceExecutedContext context)
    {
        // Empty since factories don't need to be added again (P.W. 2021-07-07 - SonarCloud issue fix)
    }
}

MultipartRequestHelper

public static class MultipartRequestHelper
{
    // Content-Type: multipart/form-data; boundary="----WebKitFormBoundarymx2fSWqWSd0OxQqq"
    // The spec says 70 characters is a reasonable limit.
    public static string GetBoundary(MediaTypeHeaderValue contentType, int lengthLimit)
    {
        var boundary = HeaderUtilities.RemoveQuotes(contentType.Boundary).Value;
        if (string.IsNullOrWhiteSpace(boundary))
            throw new InvalidDataException("Missing content-type boundary.");

        if (boundary.Length > lengthLimit)
            throw new InvalidDataException($"Multipart boundary length limit {lengthLimit} exceeded.");

        return boundary;
    }

    public static bool IsMultipartContentType(string contentType)
    {
        return !string.IsNullOrEmpty(contentType)
            && contentType.Contains("multipart/", StringComparison.OrdinalIgnoreCase);
    }

    public static bool HasFormDataContentDisposition(ContentDispositionHeaderValue contentDisposition)
    {
        // Use if you stored data objects in the body
        // Content-Disposition: form-data; name="key";
        return contentDisposition != null
            && contentDisposition.DispositionType.Equals("form-data")
            && string.IsNullOrEmpty(contentDisposition.FileName.Value)
            && string.IsNullOrEmpty(contentDisposition.FileNameStar.Value);
    }

    public static bool HasFileContentDisposition(ContentDispositionHeaderValue contentDisposition)
    {
        // Use if you stored file data in the body
        // Content-Disposition: form-data; name="myfile1"; filename="Misc 002.jpg"
        return contentDisposition != null
            && contentDisposition.DispositionType.Equals("form-data")
            && (!string.IsNullOrEmpty(contentDisposition.FileName.Value)
             || !string.IsNullOrEmpty(contentDisposition.FileNameStar.Value));
    }
}

Client

Stream _fileStream = File.OpenRead(... file ...);

        var httpClient = new HttpClient();
        httpClient.Timeout = TimeSpan.FromMinutes(30);

        var multipartContent = new MultipartFormDataContent
        {
            { new StringContent("srTest"), "Id" },
            { new StreamContent(_fileStream), "Document", "Document.doc" }
        };

        Console.WriteLine($"FileSize: {_fileStream.Length}");

        var postResponse = await httpClient.PostAsync("http://localhost:58639/api/Upload", multipartContent);

        Console.WriteLine(postResponse.StatusCode);
        Console.WriteLine(postResponse.Content.ReadAsStringAsync().GetAwaiter().GetResult());

Solution

  • Any configuration, upsetting of limits unfortunately did nothing. Had then found a post, sorry can't find it anymore, in which it is mentioned that it worked after switching to HTTPS.

    This is also the case with me. Since I run the app in HTTPS I can receive file sizes of any size. Without having changed anything else in the code.