Browser to S3 file transfer via Node

Question

Background

I want to create an input on my React/Redux SPA that takes in a file (attachment) which will then be stored in my Amazon S3 bucket.

I have the input taking a file from the local machine and sending it to my Node API fine, I have a function taking a file (local to my Node environment) and transferring it to S3 fine. I've even got both of these working together, taking a file from the local machine and writing it to the Node machine, then taking that file and transferring it to S3.

Why that doesn't work for me

The problem is the write to file/read from file time on the Node machine, AND the amount of disk used by the file being kept in temporary storage. I would like to transfer files much larger (a few gig) then the resources available to my Node instance (256m disk and even less memory). I would also like to avoid using the time and resources for the disk reads/writes.

The Question

Is there any way to get the incoming stream of information from the browser to be passed directly to S3, without ever storing it locally either on disk or in memory?

Where I am right now

app.post(`${base}/putattachment`, multer({dest: './attachments'}).single('file'), async (req, res)=> {
     const result = await Put(req.file.path, req.body.key);
     res.send(result);
 });

export const Put = (file, s3Key)=>{
    return new Promise((resolve)=>{
        const s3 = new AWS.S3();
        const params = {Bucket: BUCKET_NAME, Key: s3Key, Body: file};
        s3.putObject(params, (s3err) => {
            if (!s3err){
               resolve(s3Key);
            }
        });
    })
};

The Disclaimers

File transfer is relatively new to me. As is S3, and Express (particularly multer!). I apologize if this is straightforward for others.

Buffers, Streams, and the other related data types that are so critical here are also somewhat new to me. I'm reading as fast as I can, but I've plenty left to learn!

There will be more catches, removed for clarity given online format.

Answer 1

Multiparty library to the rescue. It is made exactly for this use case so you don't have to store data. You can just stream it.

They even have an example especially for this use case where you can directly upload images to S3. I'm not writing any code here as this example will provide you all you need to get started.