Multer returns req.file as undefined, and req.file.location as location undefined while uploading file to aws bucket

Question

I am trying to upload images to s3 Bucket. And have tried many solutions online yet I get the above errors. I don't want to store images locally, instead I want to upload them directly to s3 bucket. Any help would be appreciated.

This is Upload.js file

const AWS = require('aws-sdk');
const Keys = require('../Config/dev');
const { v4: uuidv4 } = require('uuid');
const axios = require('axios').default;
const multer = require('multer');
const multerS3 = require('multer-s3');


const s3 = new AWS.S3({
  accessKeyId: Keys.accessKeyId,
  secretAccessKey: Keys.secretAccessKey,
  region : 'ap-south-1'
});

var upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: 'thebucketname',
    acl : "public-read",
    metadata: function (req, file, cb) {
      cb(null, {fieldName: file.fieldname});
    },
    key: function (req, file , cb){
        cb(new Date().toISOString().replace(/[-T:\.Z]/g, "") + file.originalname);
    }
  })
});

module.exports = upload;

This is the router code

const express = require('express');
const Router = express.Router();
const controllers = require('../controllers/controllers.js');
const uploader = require('../controllers/Upload');
const singleUpload = uploader.single('img');

Router.post('/single-image',(req, res)=>{
    singleUpload(req, res , (err)=>{
        if(!req.file){
            console.log(req.file);
        }else
        {
        console.log(req.file);
        return res.json({'imageUrl': req.file.location});
        }
    });
});

This is how I am using postman for api request. I have also set Content-Type to Multipart/form-data inside the Headers in postman. I get the error "undefined" for req.file when I do this.

Also, If I use

 app.use(multer({dest:'./public/uploads/'}).single('file'));

my file gets stored in the 'uploads' folder but then I get the error "req.file.location undefined", and file doesn't upload to aws.

Answer 1

Firstly, if you want to upload files to s3 and not store it on your server, you can store the uploaded file as an in-memory buffer instead of writing it on your server and then uploading to s3. NOTE: this in-memory method is not recommended with large files or a large number of small files, because you need to ensure that your server has enough memory to deal with the uploads.

Then you can just pass the buffer to the s3 upload function. I don't know much about some package called multer-s3 that you've apparantly used so I'm not using that. I had made it for an array of files but it should work for single files also. I combined your code with some of my code and came up with the following:

//aws-sdk for node
const AWS = require('aws-sdk');
AWS.config.update({ region: <your region here> });

//S3
const S3 = new AWS.S3({});

const express = require('express');
const Router = express.Router();
const controllers = require('../controllers/controllers.js');
const uploader = require('../controllers/Upload');

//import multer
const multer = require("multer");



//make multer ready for in-memory storage of uploaded file
const multerMemoryStorage = multer.memoryStorage();
const multerUploadInMemory = multer({
    storage: multerMemoryStorage
});

//using multer.single as a middleware is what I prefer
Router.post('/single-image',multerUploadInMemory.single("filename"),async(req, res)=>{

    try{

        if(!req.file || !req.file.buffer){
            throw new Error("File or buffer not found");
        }

        const uploadResult = await S3.upload({
                    Bucket: "yourBucketName",
                    Key: "WhateverKeynameYouWantToGive",
                    Body: req.file.buffer,
                    ACL: 'public-read'
                }).promise();

        console.log(`Upload Successful!`);

        res.send({
            message: "file uploaded"
        })



    }catch(e){
        console.error(`ERROR: ${e.message}`);

        res.status(500).send({
            message: e.message
        })
    }

});

You can first use console.log(req.file) to see if it's not undefined (which it shouldn't be) and you can check if you're getting the buffer property in the file.

Also, it says in a "warning" here that you should never add multer as a global middleware, so app.use(multer({dest:'./public/uploads/'}) is a no-no.