Stream large blob file using StreamSaver.js

Question

I'm trying to download a large data file from a server directly to the file system using StreamSaver.js in an Angular component. But after ~2GB an error occurs. It seems that the data is streamed into a blob in the browser memory first. And there is probably that 2GB limitation. My code is basically taken from the StreamSaver example. Any idea what I'm doing wrong and why the file is not directly saved on the filesystem?

Service:

public transferData(url: string): Observable<Blob> {
    return this.http.get(url, { responseType: 'blob' });
}

Component:

download(url: string) {
    this.extractionService.transferData(url)
      .subscribe(blob => {
        const fileStream = streamSaver.createWriteStream('data.tel', {
          size: blob.size
        });
        const readableStream = blob.stream();
        if (window.WritableStream && readableStream.pipeTo) {
          return readableStream
            .pipeTo(fileStream)
            .then(() => console.log("done writing"));
        }
        const writer = fileStream.getWriter();
        const reader = readableStream.getReader();
        const pump = () =>
          reader.read()
            .then(res => res.done ? writer.close() : writer.write(res.value).then(pump));
        pump();
      });
}

The header of the requested file:
"Content-Type: application/octet-stream\r\n"
"Content-Disposition: attachment; filename=data.tel\r\n"

Answer 1

I was more or less in the same situation before. Angular http service won't work in this use case since it does not give you a ReadableStream. My solution is using fetch API instead.

But be careful that Streaming response body of fetch is an experimental feature and not compatible with all browsers. According to my test, it works fine on Google Chrome but not with Firefox or Safari. To overcome this limitation, I use a Javascript library called web-streams-polyfill together with fetch.

The code looks somehow like this:

import { WritableStream } from 'web-streams-polyfill/ponyfill';
import streamSaver from 'streamsaver';

fetch(url, {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json'
    },
    body: JSON.stringify(data)
})
.then(response => {

    let contentDisposition = response.headers.get('Content-Disposition');
    let fileName = contentDisposition.substring(contentDisposition.lastIndexOf('=') + 1);

    // These code section is adapted from an example of the StreamSaver.js
    // https://jimmywarting.github.io/StreamSaver.js/examples/fetch.html

    // If the WritableStream is not available (Firefox, Safari), take it from the ponyfill
    if (!window.WritableStream) {
        streamSaver.WritableStream = WritableStream;
        window.WritableStream = WritableStream;
    }

    const fileStream = streamSaver.createWriteStream(fileName);
    const readableStream = response.body;

    // More optimized
    if (readableStream.pipeTo) {
        return readableStream.pipeTo(fileStream);
    }

    window.writer = fileStream.getWriter();

    const reader = response.body.getReader();
    const pump = () => reader.read()
        .then(res => res.done
            ? writer.close()
            : writer.write(res.value).then(pump));

    pump();
})
.catch(error => {
    console.log(error);
});;

The idea is to check if window.WritableStream is available in the current browser or not. If not, assign the WritableStream from ponyfill directly to streamSaver.WritableStream property.

Since I faced this problem some time ago, my solution was only tested on Google Chrome 78, Firefox 70, Safari 13; web-streams-polyfill 2.0.5, and StreamSaver.js 2.0.3