Search code examples
javascriptnode.jsstreamreactive-programminghighland.js

highland.js fetch json array and transform in stream of values


I am trying to fetch a JSON array form an external API and then emit one element of the array at the time. However my implementation seems to be failing somewhere, I am getting errors instead of array

'use strict';

const request = require('request-promise'),
    H = require('highland');

H(request('http://jsonplaceholder.typicode.com/users'))
  .map(x => x.toString('utf8'))
  .tap((data) => {
      let acc = [];

      data = JSON.parse(data);
      data.forEach((entry) => {
          acc.push(entry);
      });
      return H(acc);
  })
  .each(user => console.log(user.id))// would expect that this logs 1,2,3,4
  .done(data => {
    console.log(data)
});

Solution

  • You're probably getting chunks of data rather than the full response meaning JSON.parse is attempting to parse incomplete JSON. Perhaps try something like this?

    H(request('http://jsonplaceholder.typicode.com/users'))
      .collect()
      .map(Buffer.concat)
      .flatMap(x => JSON.parse(x.toString('utf8')))
      .each(user => console.log(user.id))
      .done(data => console.log('DONE'));