How to write response to stream without waiting end of the fetch?

Question

I need to fetch an audio file and write in a directory. I am worried about that I have to wait end of the fetch to download all the file and then write/save to file. Is that possible to fetch audio file and at the same time write in a file. I just thought that it could be like "pipe". Get chunk from service and do not wait to end of the fetch process,just write chunks which are just downloaded to file. Is that possible? If yes, may you give me an example?

Here is my code:

const fetchAndWriteFiles = async (url, path) => {
  const res = await fetch(url)

  const fileStream = fs.createWriteStream(path)
  await new Promise((resolve, reject) => {
    res.body.pipe(fileStream)
    res.body.on('error', err => {
      reject(err)
    })
    fileStream.on('finish', function () {
      resolve()
    })
  })
}

Answer 1

await fetch() does not download the entire file (if it's large). It reads the first chunk of the file (in the example I looked at in the debugger, the first chunk was 16384 bytes long), extracts the headers from it and then leaves the rest of that first chunk in a stream waiting to be read out.

Subsequent parts of the file will be read after you do the .pipe() and the existing stream is emptied. The rest of the file will be read in chunks and written to disk in chunks. To test this, you'd have to use a file large enough that it will take multiple chunks to read/write it.

And, in fact, when I instrument things when downloading a larger response, I see multiple reads and writes that occur after the await fetch() is done.

So, it appears that your code is already doing what you seemed to be interested in. If the page you're downloading is small (fits inside of one buffer), you won't see the multiple reads/writes because the whole response may be read in one pass, but for larger responses, it will definitely go in chunks. The size of the chunks may also depend upon how fast the data arrives from the source.