Asp.Net Core File Upload on AWS ECS/EC2 in Container

Question

I have an asp .net core application deployed to AWS in a docker linux container, hosted in ECS into an EC2 server (not fargate). We have a react front-end UI that uploads a file to the .net core api, which we then process. This works properly in dev, however, in prod, the file upload is failing and I think that the server does not have access/rights to save the file to our upload folder (think webapproot/upload).

Do we need to take the file, save it to S3, then process from there? or is this simply a rights issue?

Answer 1

We got this to work by not saving it in-flight, we kept it as a byte stream until we stored it in the S3 bucket. Note, you'll want to make sure you use IFormFileCollection and ensure when you start the app that you include your secrets, and in your CI/CD you tell aws to include secrets from Secrets Manager (assuming you're hosting them there).

Endpoint:

    [HttpPost]
    [Consumes("multipart/form-data")]
    [RequestSizeLimit(int.MaxValue)]
    public async Task<IActionResult> Post([FromForm]IFormFileCollection files, [FromForm]FileTypes fileType)
    {
        try
        {
            long size = files.Sum(f => f.Length);

            var fileName = string.Empty;

            foreach (var formFile in files)
            {
                if (formFile.Length > 0)
                {
                    fileName = $"{fileType.ToString()}-{Guid.NewGuid().ToString()}";

                    await _mediator.Send(new S3Messages.Save(formFile.OpenReadStream(), "fileimport", fileName));
                }
            }

            return Ok(new FileUploadResponse
            {
                Count = files.Count,
                Size = size,
                Files = files.Select(f => new ImportFile()
                {
                    Created = DateTime.Now,
                    FileName = fileName,
                    FileSize = size,
                    FileTypeID = (int)fileType,
                }),
            });
        }
        catch (Exception ex)
        {
            return BadRequest(ex.Message);
        }
    }

Program.cs:

      public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .ConfigureAppConfiguration((context, config) =>
                {
                    config.AddUserSecrets<Startup>();

                    if (context.HostingEnvironment.IsProduction())
                    {
                        config.AddEnvironmentVariables();

                        var secretsJson = Secrets.Get("us-east-1");

                        var dict = JsonConvert.DeserializeObject<Dictionary<string, string>>(secretsJson);

                        config.AddInMemoryCollection(dict);
                    }
                })
                .UseServiceProviderFactory(new ServiceProviderFactory<Program>())
                .ConfigureWebHostDefaults(builder =>
                {
                    builder
                        .UseUrls("http://*:5000")
                        .UseStartup<Startup>();
                });

ci/cd pipeline yaml file, ensure to bring the secrets in when creating the task:

- export TASK_VERSION=$(aws ecs register-task-definition --family "${ECS_TASK_NAME}" --network-mode host --execution-role-arn "xxx" --container-definitions "[{\"name\":\"$PROJECT_NAME\",\"image\":\"$IMAGE_NAME\",\"portMappings\":[{\"containerPort\":5000,\"hostPort\":5000,\"protocol\":\"tcp\"}],\"memoryReservation\":512,\"memory\":2004,\"essential\":true,\"environment\":[{\"name\":\"SECRETS_NAME\",\"value\":\"$xxx_SECRETS_NAME\"}],\"secrets\":[{\"name\":\"$xxx_SECRETS_NAME\",\"valueFrom\":\"$xxx_SECRETS\"}]}]" | jq --raw-output '.taskDefinition.revision')