Search code examples
javascriptdownloadfileapi

Download big files

I have a file represented as a list of chunks, and the goal is to download all chunks, join and save as a file.

Requirements

  1. It should work for large files
  2. It should be cross-browser solution

What I've found...

  1. Use JS Array
    Yes, we can download and store all chunks in regular Javascript array.
    • It's cross-browser solution
    • But it uses RAM, and if file size exceeds free memory browser just crashes...
  2. FileSaver.js
    • Partly cross-browser
    • Limited file size
  3. StreamSaver.js
    • Not cross-browser
    • Works for large files
  4. Filesystem API
    • It's Chrome sandbox filesystem api
    • Works for large files

But I still can't achieve my goal with covered requirements...
If someone has experience for best solution I kindly ask to share it here. Thanks

Solution

  • There isn't really a cross-browser option here yet unfortunately.

    In Chrome, you can use either the non-standard Filesystem API, or Blobs which Chrome will use the file-system for if the blob is large.

    In Firefox, you can use maybe use the non-standard IDBMutableFile. However, it will not work with the download API, so you would have to use window.location to send the browser to the blob URL, which the browser must then download (may not happen for all file extensions). You also may need to use the IDB persistent option to have files larger than ~2GB.

    In other browsers, Blob is your only real option. On the up side, the OS the browser runs on may use paging which could enable the browser to create blobs larger than memory.

    A service-worker-based option like StreamSaver may also help (perhaps this could be a download API alternative for Firefox), but there is (or was?) a limit to how long the browser will wait for a complete response, meaning you would probably have to download and store the chunks somewhere to complete the response in time.