Node fetch receive a ReadableStream and receive an incomplete response

Question

I am making a request Node fetch receive a ReadableStream and receive an incomplete response. The problem seen as the ReadableStream is not getting complete in the await.

Request:

static async postData(url = "") {
    // Default options are marked with *
    const response = await fetch(url, {
      method: "POST", // *GET, POST, PUT, DELETE, etc.
      mode: "same-origin", // no-cors, *cors, same-origin
      cache: "default", // *default, no-cache, reload, force-cache, only-if-cached
      credentials: "same-origin", // include, *same-origin, omit
      headers: {
        "Content-Type": "application/json",
        // 'Content-Type': 'application/x-www-form-urlencoded',
      },
      redirect: "follow", // manual, *follow, error
      referrerPolicy: "no-referrer", // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
      //body: JSON.stringify(dados), // body data type must match "Content-Type" header
    });
    const stream = await response.body?.getReader().read();

    let jsonBuffer = Buffer.from(stream?.value!);

    let jsonString = jsonBuffer.toString("utf8");
    console.log(jsonString);
    return JSON.parse(jsonString); // parses JSON response into native JavaScript objects
  }

Response:

{"retorno":{"status_processamento":"3","status":"OK","pagina":1,"numero_paginas":1,"contatos":[{"contato":{"id":"715461091","codigo":"","nome":"Fabio Moreno","fantasia":"","tipo_pessoa":"F","cpf_cnpj":"","endereco":"","numero":"","complemento":"","bairro":"Vila Medon","cep":"","cidade":"Americana","uf":"SP","email":"[email protected]","fone":"","id_lista_preco":0,"id_vendedor":"0","nome_vendedor":"","s`

Error:

[1] SyntaxError: Unexpected end of JSON input
[1] at JSON.parse ()
[1] at TinyERP.postData (file:///home/linkiez/Desktop/Projetos/JCMserver3/dist/services/tinyERP.js:22:21)
[1] at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[1] at async aprovarOrcamento (file:///home/linkiez/Desktop/Projetos/JCMserver3/dist/controllers/orcamentoController.js:259:40)
[1] nodemon --experimental-specifier-resolution=node -q dist/index.js exited with code SIGINT
[0] tsc --watch exited with code SIGINT

Answer 1

You've said you're using Node.js's fetch, which is meant to be compatible with the web platform's fetch.

Your code isn't reading the entire response. Here's the documentation for the read() method on the default reader returned by getReader() with no arguments:

The read() method of the ReadableStreamDefaultReader interface returns a Promise providing access to the next chunk in the stream's internal queue.

(my emphasis) That's not the entire response, that's just the first chunk of the response.

But there's no need for that code to be anywhere near that complicated, just use the built-in json method to read the entire response and parse it from JSON; see the *** comments below:

static async postData(url = "") {
    const response = await fetch(url, {
        method: "POST",
        mode: "same-origin",
        cache: "default",
        credentials: "same-origin",
        headers: {
            "Content-Type": "application/json",
        },
        redirect: "follow",
        referrerPolicy: "no-referrer",
        // *** It seems odd that there's no `body` here, given it's a POST
        // saying that it's *sending* JSON (the `Content-Type` header above
        // says what you're *sending*, not what you're expecting back).
    });
    // *** This was missing, but it's important; `fetch` only rejects on
    // *network* errors, not HTTP errors:
    if (!response.ok) {
        throw new Error(`HTTP error ${response.status}`);
    }
    // *** Fully read the response body and parse it from JSON:
    return await response.json();
}

Here's a post on my anemic old blog about the need for the ok check I added above.