fetch response is not of same size for same URL

Question

I am using fetch API to get a response from the server URL and the response size is not every time.

Most of the time I get response data of size 262144 but sometimes the size is less than that. Like data of size 65536 and 196608.

async function fetchData() {
  let url = "https://www.dl.dropboxusercontent.com/s/7d87jcsh0qodk78/fuel_64x64x64_uint8.raw?dl=1";
  let response = await fetch(url);
  let data = await response.body.getReader().read();
  data = data.value;
  if (data) {
    dataBuffer = new Uint8Array(data);
    console.log(data.length);
  } else {
    console.log("action aborted");
  }
}

fetchData()

Answer 1

The reader you receive from getReader() works with an internal queue. The documentation of read() says the following:

The read() method of the ReadableStreamDefaultReader interface returns a promise providing access to the next chunk in the stream's internal queue.

Fetching multiple times may end up chunking the remote data differently, which in turn results in the different lengths. To read the stream to completion check the done value returned by read().

async function fetchData() {
  const url = "https://www.dl.dropboxusercontent.com/s/7d87jcsh0qodk78/fuel_64x64x64_uint8.raw?dl=1";
  const response = await fetch(url);
  const reader = response.body.getReader();
  
  let length = 0;
  let value, done;
  while ({value, done} = await reader.read(), !done) {
    length += value.length;
  }
  console.log(length);
}

fetchData()

However if the intent is to read the stream to completion before taking any action, you might as well use one of the response methods like arrayBuffer(), blob(), formData(), json() or text() depending on the type of data you are expecting. The mentioned response methods all read the response stream to completion.

async function fetchData() {
  const url = "https://www.dl.dropboxusercontent.com/s/7d87jcsh0qodk78/fuel_64x64x64_uint8.raw?dl=1";
  const response = await fetch(url);
  const dataBuffer = new Uint8Array(await response.arrayBuffer());
  
  console.log(dataBuffer.length);
}

fetchData()