"Error: MultipartParser.end(): stream ended unexpectedly" error when uploading a file

I'm uploading a file from a Buffer using form-data. On the server side I'm using formidable to parse the file data. I keep getting errors like this on the server:

Error: MultipartParser.end(): stream ended unexpectedly: state = START

Error: MultipartParser.end(): stream ended unexpectedly: state = PART_DATA

I'm not sure if this is a problem with form-data or formidable. I found a lot of solutions (mostly involving not setting the Content-Type header manually). However, I couldn't find one that resolved the issue for me. I ended up figuring something out, so posting in order to answer.

I encountered this issue while developing a Strapi upload provider. Strapi provides information about a file that needs to be uploaded to a service. The file contents are provided as a Buffer (for some reason). Here's what my code looked like when I was getting the error (modified slightly):

const form = new FormData()
form.append('file', Readable.from(file.buffer))
form.append('name', file.name)
form.append('hash', file.hash)
form.append('mime', file.mime)
form.on('error', () => abortController.abort())

return fetch(url, {
    method: 'post',
    body: form,
    signal: abortController.signal,
}))

Solution

Again, I'm not sure if this is a problem with form-data or formidable, but if I provide a filename and knownLength to form-data, the issue goes away. This is what my final code looks like (modified slightly):

const fileStream = Readable.from(file.buffer)
const fileSize = Buffer.byteLength(file.buffer)

const abortController = new AbortController()

const form = new FormData()
form.append('file', fileStream, {filename: file.name, knownLength: fileSize})
form.append('name', file.name)
form.append('hash', file.hash)
form.append('mime', file.mime)
form.on('error', () => abortController.abort())

return fetch(url, {
    method: 'post',
    body: form,
    signal: abortController.signal,
}))

I've tried logging the form.on('error') result and I get nothing (it's not aborting).

I've tried just setting filename and I get the same error.

I've tried just setting knownLength. The file uploads but it's empty (at least, formidable thinks it is). It must need the filename in order to parse the file correctly?

This is likely an issue with form-data not reading the input stream properly or not writing to the output stream properly (I did notice looking at the raw form data on the server that the file data was truncated) or with formidable not reading the file data properly. There's something about setting the filename and knownLength that bypasses the issue.

UPDATE: This may have been partially fixed in a newer version of form-data. I updated the package and no longer need to set the knownLength. I still need to set filename though. Without it, the server thinks the file is empty.