I'm trying to add a simple file upload API endpoint to my Express+no-stress+Typescript app, so I started with what seemed to be the easy way, i.e. express-fileupload.
I basically resigned on the spot, as soon as I realized it doesn't play well with Typescript, because of this way of accessing the uploaded file, which Typescript doesn't seem to like very much (the variable named after my <input> HTML element is unknown to Typescript so it does not even compile the code). There already exists another question for this problem, which did not receive any proper answer, but the accepted non-answering answer suggests to move to multer, and that's what I did too.
So I ran npm uninstall express-fileupload followed by npm install multer. I tried several combinations of the multer middleware (and even without any middlewares, but calling multer directly in the /upload POST callback), but I always got the "Unexpected end of form" error. This went on until I found this open issue and tried downgrading multer to 1.4.3, which did work to some extent (I started getting different errors), but in the end it wasn't an acceptable solution for me, because of the security implications of that downgrade.
Then I moved to busboy. I still got the same "Unexpected end of form" error. Googling that with "busboy" added, landed me to this fairly recent unanswered question, where the best you can find is a comment telling that both multer and express-fileupload use busboy behind the scenes and suggesting to use one of them instead...
So I moved back to multer, because clearly it is not multer to blame and multer seems to be what most devs out there actually use. Here below is the relevant code I've written. It's part of a minimal Express+no-stress+Typescript app I generated from scratch to have a minimal reproducible example. However the generated scaffold is not minimal enough to post here the whole thing, so I put here only what I modified, the rest is just what you can get by generating a new Express+no-stress+Typescript project or by clonig my sample repo.
/server/api/controllers/examples/router.ts file:
/* eslint-disable prettier/prettier */
import express from 'express';
import controller from './controller';
export default express
.Router()
.post('/:dir', controller.post.bind(controller));
/server/api/controllers/examples/controller.ts file:
/* eslint-disable prettier/prettier */
import { Request, Response } from 'express';
import path from 'path';
import multer from 'multer';
export class Controller {
filename(_req: Request, file: Express.Multer.File, cb: (error: Error|null, name: string) => void) {
console.log("file will be named " + file.originalname);
cb(null, file.originalname);
}
post(req: Request, res: Response): void {
console.log("Received POST request");
const storage = multer.diskStorage({
destination: path.join('/tmp', req.params.dir), // Trusting req.params.dir is insecure, but this is only an example
filename: this.filename
});
const upload = multer(
{ storage: storage,
limits: { fieldNameSize: 1000000,
fieldSize: 1000000
}
}).single('doc');
console.log("Multer is set to receive a single 'doc' upload");
upload(req, res, (err: any) => {
if (err) {
console.log("Upload failed with error: " + err.message);
res.status(422);
res.json(err.message);
res.end();
}
});
}
}
export default new Controller();
and the /server/common/api.yml file, not really needed for Multer nor Express, but it enables you to test the file upload endpoint with Swagger (http://localhost:3000)
openapi: 3.0.1
info:
title: multertest
description: Multer Test
version: 1.0.0
servers:
- url: /api/v1
tags:
- name: Examples
description: Simple example endpoints
- name: Specification
description: The swagger API specification
paths:
/examples/{dir}:
post:
tags:
- Examples
description: Uploads a file
parameters:
- name: dir
in: path
description: the subdirectory where the uploaded file will be stored
required: true
schema:
type: string
requestBody:
content:
multipart/form-data:
schema:
type: object
properties:
doc:
type: string
format: binary
required: true
responses:
200:
description: ok
content: {}
/spec:
get:
tags:
- Specification
responses:
200:
description: Return the API specification
content: {}
Finally, here is the /server/common/server.ts file, which I did NOT modify, but someone suggested it might be the cause of the problem. I did try setting higher upload limits and uploading files way smaller than the limits set, but nothing changed.
import express, { Application } from 'express';
import path from 'path';
import bodyParser from 'body-parser';
import http from 'http';
import os from 'os';
import cookieParser from 'cookie-parser';
import l from './logger';
import errorHandler from '../api/middlewares/error.handler';
import * as OpenApiValidator from 'express-openapi-validator';
const app = express();
export default class ExpressServer {
private routes: (app: Application) => void;
constructor() {
const root = path.normalize(__dirname + '/../..');
app.use(bodyParser.json({ limit: process.env.REQUEST_LIMIT || '100kb' }));
app.use(
bodyParser.urlencoded({
extended: true,
limit: process.env.REQUEST_LIMIT || '100kb',
})
);
app.use(bodyParser.text({ limit: process.env.REQUEST_LIMIT || '100kb' }));
app.use(cookieParser(process.env.SESSION_SECRET));
app.use(express.static(`${root}/public`));
const apiSpec = path.join(__dirname, 'api.yml');
const validateResponses = !!(
process.env.OPENAPI_ENABLE_RESPONSE_VALIDATION &&
process.env.OPENAPI_ENABLE_RESPONSE_VALIDATION.toLowerCase() === 'true'
);
app.use(process.env.OPENAPI_SPEC || '/spec', express.static(apiSpec));
app.use(
OpenApiValidator.middleware({
apiSpec,
validateResponses,
ignorePaths: /.*\/spec(\/|$)/,
})
);
}
router(routes: (app: Application) => void): ExpressServer {
routes(app);
app.use(errorHandler);
return this;
}
listen(port: number): Application {
const welcome = (p: number) => (): void =>
l.info(
`up and running in ${
process.env.NODE_ENV || 'development'
} @: ${os.hostname()} on port: ${p}}`
);
http.createServer(app).listen(port, welcome(port));
return app;
}
}
So the question is: how do I get a simple file upload to work with my "Express no stress typescript" application?
The issue with your code was that you were encountering a conflict between the express-openapi-validator middleware and multer while handling formdata. This conflict arose because the express-openapi-validator middleware uses multer to handle the formdata bodies, and the multer library was also being used in your code to handle the same data.
However, the good news is that the uploaded files are already available in req.body.files in your express route, as express-openapi-validator uses multer to handle the files.
To resolve the issue, you can simply use multer disk storage to handle the uploaded files. The disk storage is used to specify the destination directory and the filename for the uploaded files. The code below is a working version of the controller from your example repository:
/* eslint-disable prettier/prettier */
import {
Request,
Response
} from 'express';
import path from 'path';
import multer from 'multer';
import stream from 'stream';
export class Controller {
filename(
_req: Request,
file: Express.Multer.File,
cb: (error: Error | null, name: string) => void
) {
console.log('file will be named ' + file.originalname);
cb(null, file.originalname);
}
post(req: Request, res: Response): void {
console.log('Received POST request');
const storage = multer.diskStorage({
destination: path.join('tmp', req.params.dir), // Trusting req.params.dir is insecure, but this is only an example
filename: this.filename,
});
(req.files as Array < Express.Multer.File > ).forEach(
(file: Express.Multer.File) => {
storage._handleFile(
req, {
...file,
stream: new stream.PassThrough(),
},
(err: any) => {
if (err) {
console.log('Upload failed with error: ' + err.message);
res.status(422);
res.json(err.message);
res.end();
}
}
);
}
);
res.status(200);
res.json('OK');
res.end();
}
}
export default new Controller();