How to Store and Access Large Files in NodeJS Web Application

Question

If I want to allow a user to upload a video in a post, then what's the best way to store it so that I can access it later? I am currently just using a INSERT query to upload it to a PostGres SQL database field, but it's extremely slow.

Is there another method or best practice to store large files like this and access it quickly and efficiently?

var video = videoFile ? "'\\x" + videoFile.buffer.toString('hex') + "'" : "NULL";
let insertQuery = "INSERT INTO posts (video) VALUES(" + video + ") RETURNING *";

Above is the code I'm using right now to insert the file into the database.

Answer 1

Do not store the whole video into the database. These 2 steps below might be a better solution:

1) Upload the video file to the cloud storage. See Google Cloud Strage and Amazon S3. There are many other options as well. Go do some comparison yourself.

2) You will have a link or ID (or key in S3) to access a file. Store all those links or ID into your database. Please take a look at this GCS doc and this S3 question.

Also, make use of the non-blocking I/O which is a good thing of node.js. In your case, you may try something like this:

let insertQuery;
let promisesList=[];

for(let vid of videoIds){
    insertQuery = "INSERT INTO posts (videoId) VALUES($1) RETURNING *";
    promisesList.push(client.query(query,[vid]));
}
await Promise.all();