Unable to consume ndjson stream in Node, but similar code works in React

Question

I am trying to consume data from this endpoint on lichess.org.

Here is a minimal working example of a React component consuming that stream of data. I'm using a library called can-ndjson-stream.

import ndjsonStream from "can-ndjson-stream"
import { useEffect } from "react"

function App() {
  useEffect(() => {
    fetch("https://lichess.org/api/tv/feed")
      .then(res => ndjsonStream(res.body))
      .then(stream => {
        const streamReader = stream.getReader()
        streamReader.read().then(async res => {
          while (!res || !res.done) {
            res = await streamReader.read()
            console.log(res.value)
          }
        })
      })
      .catch(console.error)
  }, [])
  return <>Lorem Ipsum</>
}

export default App

However, if I try to write this same code and run it in Node like this:

import ndjsonStream from "can-ndjson-stream"
import fetch from "node-fetch"

fetch("https://lichess.org/api/tv/feed")
  .then(res => ndjsonStream(res.body))
  .then(stream => {
    const streamReader = stream.getReader()
    streamReader.read().then(async res => {
      while (!res || !res.done) {
        res = await streamReader.read()
        console.log(res.value)
      }
    })
  })
  .catch(console.error)

I get this error:

ReferenceError: ReadableStream is not defined at ndjsonStream

So it seems like the res from the fetch is null or undefined, but fetching other APIs works fine.

I also tried using axios instead of node-fetch like this:

import ndjsonStream from "can-ndjson-stream"
import axios from "axios"

axios
  .get("https://lichess.org/api/tv/feed")
  .then(res => ndjsonStream(res.data))
  .then(stream => {
    const streamReader = stream.getReader()
    streamReader.read().then(async res => {
      while (!res || !res.done) {
        res = await streamReader.read()
        console.log(res.value)
      }
    })
  })
  .catch(console.error)

But it just hangs and shows no output. Thanks to anyone that could shed some light on this or offer any alternative way to run this in Node.

Answer 1

Thanks to the comments from tromgy, I was able to make something that works. I went with the library hyperquest to help handle the request and piping of the stream. I also used the ndjson library.

Here is some working code:

hyperquest("https://lichess.org/api/tv/feed")
    .pipe(ndjson.parse())
    .on("data", console.log)

Note that you can manipulate the objects as they arrive with the second argument of on(), like below.

...
.on("data", (obj) => {
    foo(obj)
})