Memory usage of ArrayBuffer and Blob objects

Question

I'm getting ArrayBuffer chunks of data and I am adding each chunk to an array on a web worker and when I get all the chunks I convert that array of chunks to a blob.

//worker.js

const array = []

this.onmessage = function(e){
     if(e.data.done)
     {
        const blob = new Blob(array);
        this.postMessage(blob)
        shunks.length = 0
     }
     else
     {
         array.push(e.data.chunk)
     }
}

MY QUESTIONS ARE

1. if the array size hits 2GB it will be stored on the memory right? that means I wont be able to fill that array with data greater than my available memory?
2. When I create a blob from that array, will the blob also take another 2GB from the memory?

Answer 1

That depends...

Theoretically, yes each ArrayBuffer will occupy its byteLength in the memory, and yes, Blobs made from it will occupy new space in memory.

However browsers may use some optimizations here, for instance IIRC Chrome does save its Blobs on the user's disk instead of bloating the memory, and I believe they could use similar tricks for ArrayBuffers.

So when possible, it may be preferable to build a Blob per smaller chunk (since each "chunk" Blob would be saved on the disk), but that's quite unsure and relies on strong assumptions. If you need to be safe, better assume it will be on the memory and that you are at risk of reaching a limit.

If you really need to handle huge files, you may consider Streams, for instance the new File System Access API allows us to write to disk as streams. On systems where this is not accessible, you could try saving each chunk in IndexedDB.