Express Route returns status code 500 with generic 'Error' when file is sent as 'multipart/form-data'

Question

DETAILS:

I have an express route which is set up to accept files as multipart/formdata and upload them to an S3 bucket. I am using multer to filter image types, as well as store them temporarily on the server through the creation of an upload directory. The files are removed shortly after upload success. The array of files are named images as per multer configuration, and accepts a maximum of 3 images.

The code works perfectly on my local machine. I test through POSTMAN and can upload 1-3 files and get the proper response. If there are no files attached, the correct response is triggered as well, all with status code 200.

PROBLEM:

The exact same codebase is deployed on Amazon ECS with Docker, but somehow keeps failing consistently with status code 500 and a generic 'Error' message that is not found in the codebase. Using logs I have determined that multer is not the cause, as it passes through the filter. It appears to be failing somewhere between the multer middleware and the route itself, with an exception.

Exception: Using POSTMAN, if a multipart/formdata POST request is made with no files I.E empty images array, the route is triggered properly and the message "You did not attach any images" is returned as a response.

I have been unable to figure out the issue and appreciate it if some guidance can be provided on this issue!

CODE SNIPPETS:

filesController:

files.post(
  "/multiple",
  upload.array("images", 3),
  async (req: ExpressRequest, res: ExpressResponse) => {
    try {
      const files: { [fieldname: string]: Express.Multer.File[] } | Express.Multer.File[] =
        req.files;
      console.log("FILES", files);
      // execute only if there are files
      if (files.length > 0) {
        const dataPromises = (files as Array<Express.Multer.File>).map(
          async (file: Express.Multer.File) => {
            // check if file.mimetype here is 'image/heic', and convert into jpeg accordingly
            const fileNameWithoutExt = file.filename.split(".")[0];
            try {
              if (file.mimetype == "image/heic") {
                await convertFile(file, fileNameWithoutExt, 0.2);
                const response = await uploadFilePath(
                  S3_IMAGE_BUCKET,
                  `./uploads/${fileNameWithoutExt}.jpeg`,
                  `${fileNameWithoutExt}.jpeg`
                );
                console.log("HEIC File Upload Response", response);
                fs.unlinkSync(`./uploads/${fileNameWithoutExt}.jpeg`);
                fs.unlinkSync(file.path);
                return {
                  fileName: `${fileNameWithoutExt}.jpeg`,
                  metaData: response.$metadata,
                };
              } else {
                const response = await uploadFile(S3_IMAGE_BUCKET, file);
                console.log("JPEG File Upload Response", response);
                fs.unlinkSync(file.path);
                return {
                  fileName: file.filename,
                  metaData: response.$metadata,
                };
              }
            } catch (err) {
              console.error("Error for file conversion/upload", err, err.stack);
              res.status(500).send({
                message: "Upload failed due to conversion or something.",
                error: err,
                stack: err.stack,
              });
            }
          }
        );
        const fileData = await Promise.all(dataPromises);
        const fileNames = fileData.map((data: any) => data.fileName);
        const statusCodes = fileData.map((data: any) => data.metaData.httpStatusCode);

        if (statusCodes.find((statusCode) => statusCode === 200)) {
          res.status(200).send({
            filePath: `/image/`,
            fileNames,
          });
        } else {
          res.status(403).send({
            message: "Upload failed. Please check credentials or file has been selected.",
          });
        }
      } else {
        res.status(200).send({
          message: "You did not attach any images",
        });
      }
    } catch (err) {
      res.status(500).send({
        message: "Upload failed. Please check credentials or file has been selected.",
      });
    }
  }
);

multer configuration:

const storage = multer.diskStorage({
  // potential error, path to store files, callback
  destination: (req, file, cb) => {
    // cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
    cb(null, "uploads/");
  },
  filename: (req, file, cb) => {
    console.log("MULTER STORAGE STARTED")
    const date = new Date().toISOString().substring(0, 10);
    // const name = `${req.body.first_name}_${req.body.last_name}`;
    // cb defines the name of the file when stored
    const alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz-";
    const nanoid = customAlphabet(alphabet, 20);
    cb(null, `${date}_${nanoid()}_${file.originalname}`);
    console.log("FILE NAME CREATED, MULTER STORAGE STOPPED")
  },
});

/* Accept jpeg or png files only */
// NOTE: file type rejection works, but there is no error message displayed if file is rejected. logic in route continues to be executed
const fileFilter = (
  req: Request,
  file: Express.Multer.File,
  cb: (error: Error | null, accepted: boolean) => void
) => {
  console.log("======== FILE FILTER ========", file);
  if (
    file.mimetype === "image/jpeg" ||
    file.mimetype === "image/png" ||
    file.mimetype === "image/heic"
  ) {
    cb(null, true);
    console.log("FILTER PASSED")
  } else {
    console.log("FILTER FAILED");
    cb(null, false);
  }
};

/* Only accepts filesize up to 5MB */
// the first parameter is super important that determines where the data is stored on the server
const upload = multer({
  dest: "uploads/", // default simple config to upload file as binary data
  storage, // enable if storage of actual file is required.
  // limits: { fileSize: 1024 * 1024 * 5 },
  fileFilter,
});

SCREENSHOTS:

response with no images in form data

response with images in form data

Answer 1

Can you make sure the upload directory exists in your Docker container? Multer will not create it if it doesn't exist. It might be failing silently between your storage function and the actual writing of the files to disk.

const storage = multer.diskStorage({
  // potential error, path to store files, callback
  destination: (req, file, cb) => {
    // cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
    cb(null, "uploads/");
  },

should be something like:

import { access, constants } from "fs";
import { join } from "path";
...
const storage = multer.diskStorage({
  // potential error, path to store files, callback
  destination: (req, file, cb) => {
    // cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
    const currentDirectory: string = process.cwd();
    const uploadDirectory: string = join(currentDirectory, 'uploads/');
    // can we write to this path?
    access(uploadDirectory, constants.W_OK, (err) => {
      if (err) {
        console.error(err);
        cb(err, null);
      }
      cb(null, uploadDirectory);
    })
  },