Heroku and multer

I made a webapp that lets you upload a file to the server using multer. It works like it should when the server is run locally, but when I deployed it on Heroku, it seems I ran into a 500 internal server error.

Has anyone dealt with this before?

What are the options?

Webapp is here: https://dupefinder.herokuapp.com/

Github repo is here: https://github.com/ExtDASH/herkodeploy

2018-09-19T19:38:48.310177+00:00 app[web.1]: POST /uploads 500 148 - 181.170 ms
2018-09-19T19:38:48.310830+00:00 app[web.1]: Error: ENOENT: no such file or directory, open 'uploads/csv1537385928295.csv'
2018-09-19T19:38:48.311255+00:00 heroku[router]: at=info method=POST path="/uploads" host=dupefinder.herokuapp.com request_id=ff1aaa34-f36c-49cf-bd4e-4a936fb48a2c fwd="24.52.32.175" dyno=web.1 connect=1ms service=188ms status=500 bytes=404 protocol=https

and here is the console error in the browser:

main.js:146 POST https://dupefinder.herokuapp.com/uploads 500 (Internal Server Error)
reader.onload @ main.js:146
load (async)
readFile @ main.js:131
invoker @ vue.js:2029
Vue.$emit @ vue.js:2538
click @ VBtn.ts:108
invoker @ vue.js:2029
fn._withTask.fn._withTask @ vue.js:1828

I'm using an XMLHttpRequest POST request:

readFile: function(){
        const input = document.querySelector('#myFile')
        const reader = new FileReader()
        reader.onload = function() {
            let csvfile = new Blob([reader.result], { type: 'text/csv' })
            app.uploadingFile = true

            const form = new FormData()
            let sendName = input.files[0].name.split(/\W+/g)

            form.append('Ncsv', csvfile, `${sendName[0]}.csv`)
            const xhr = new XMLHttpRequest()
            xhr.open('POST', '/uploads', true)
            xhr.onreadystatechange = function() {
                if(this.readyState == XMLHttpRequest.DONE && this.status == 200) {
                    form.delete('Ncsv')
                }
            }
            xhr.send(form)

        }
        reader.readAsText(input.files[0])

which goes to an app.post route in my server.js file:

const express = require('express')
const connect = require('connect')
const morgan = require('morgan')
const bodyParser = require('body-parser')
const mongoose = require('mongoose')
const fs = require('fs-extra')
const multer = require('multer')
const getRouter = require('./routes/ourNums')
const nFs = require('./fileSchema.js')
const namesRouter = require('./routes/namesRouter.js')
const computeRouter = require('./routes/computeRouter.js')

// const uploadRouter = require('./routes/uploadRouter') unused for now
const filesRouter = require('./routes/filesRouter')
const path = require('path')



const app = express()

var storage = multer.diskStorage({
destination: function (req, file, cb) {
    fs.ensureFile(file)
    .then(() => {
        console.log('done')
     }
    cb(null, __dirname+'/uploads')
},
filename: function (req, file, cb) {
    var newName = file.originalname.split(/\W+/g)
    var fullName = `${newName[0]}${Date.now()}.csv`
    cb(null, fullName)
},
})



var upload = multer({ storage: storage })

app.use(express.json({limit: '50mb'}))
app.use(bodyParser.urlencoded({extended: false}))
app.use(bodyParser.json())
app.use(bodyParser.json({ limit: '50mb' }))
app.use(morgan('tiny')) //watching for changes
// app.use(express.static(`${__dirname}/client/index.html`))

app.post('/uploads', upload.single('Ncsv'), function (req, res, next) {
var fileName = req.file.filename
nFs.create({
    name: fileName
})
.then(data => res.status(200).send())
.catch(e => {
    req.error = e
    console.log(e)
    next()
})
})

before, I didn't use fs for anything (and as you can see here, fs.ensureFile isn't doing anything to fix the 500 error), I just included it to begin with so I could play around with it. Running the server locally, this works. I click my upload button on my client and it sends whatever file i selected as a blob, runs it through multer, and creates the file in a /server/uploads/ directory

Edit: I just tried using multer.memoryStorage() and got the same 500 internal server error.

Solution

I'm not sure why your uploads aren't being saved; you should be able to save them temporarily.

But this won't work long-term. Heroku's filesystem is ephemeral: any changes you make will be lost the next time your dyno restarts, which happens frequently (at least once per day).

Heroku recommends storing uploads on something like Amazon S3. Here's a guide for doing it specifically with Node.js.

Once you've stored your files on S3 you should be able to retrieve them using an appropriate library or possibly over HTTP, depending on how you've configured your bucket.

If you still wish to use multer, check out multer-s3.