Search code examples
node.jsamazon-web-servicesamazon-s3aws-lambdakoa

AWS lambda function issue with FormData file upload

I have a nodejs code which uploads files to S3 bucket. I have used koa web framework and following are the dependencies:

    "@types/koa": "^2.0.48",
    "@types/koa-router": "^7.0.40",
    "koa": "^2.7.0",
    "koa-body": "^4.1.0",
    "koa-router": "^7.4.0",

following is my sample router code:

import Router from "koa-router";

const router = new Router({ prefix: '/' })
router.post('file/upload', upload)

async function upload(ctx: any, next: any) {
    const files = ctx.request.files
    if(files && files.file) {
      const extension = path.extname(files.file.name)
      const type = files.file.type
      const size = files.file.size
      console.log("file Size--------->:: " + size);
      sendToS3();
    }
  }

function sendToS3() {
   const params = {
      Bucket: bName,
      Key: kName,
      Body: imageBody,
      ACL: 'public-read',
      ContentType: fileType
    };

   s3.upload(params, function (error: any, data: any) {
      if (error) {
        console.log("error", error);
        return;
      }
      console.log('s3Response', data);
      return;
    });
}

The request body is sent as FormData.

Now when I run this code locally and hit the request, the file gets uploaded to my S3 bucket and can be viewed. In Console the file size is displayed as follows: filesize---->32212 which is the correct actual size of the file.

But when I deploy this code as lambda function and hit the request then I see that the file size has suddenly increased(cloudwatch log screenshot below). filesize---->51619

Still that file gets uploaded to S3 but the issue is when I open the file it show following error. enter image description here

I further tried to find whether this behaviour persisted on standalone instance on aws. But it did not. So the problem occurs only when the code is deployed as a serverless lambda function.

I tried with postman as well as my own front end app. But the issue remains.

I don't know whether I have overlooked any configuration when setting up the lambda function that handles such scenarios.

This is an unprecedented issue I have encountered, and really would want to know if any one else encountered same before. Also I am not able to debug and find why the file size is increasing. I can only assume that when the file reaches the service, some kind of encoding/padding is being done on the file.

Solution

  • Finally was able to fix this issue. Had to add "Binary Media Type" in AWS API Gateway

    Following steps helped.

    AWS API Gateway console -> "API" -> "Settings" -> "Binary Media Types" section.

    Added following media type:

    multipart/form-data

    Save changes.

    Deploy api.

    Info location: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings-configure-with-console.html