Search code examples
node.jsfilestream

Create Read Stream from remote location in node

I'm trying to pull a csv file from a remote url in node and use csvtojson to parse it. I am using the request library to get the file but cannot turn it into a readable stream so that I can send it into csvtojson. Sorry this is typescript but it should be interpretable. If I do the following

request.get("http://myurl.com", (err, response, body) => {
    console.log(body);
});

Something that looks like the file gets printed out but I can't turn it into a writable stream. It certainly isn't anything I can send into csvtojson.
I know you can create a writestream like this. 

request('http://google.com/doodle.png').pipe(fs.createWriteStream('doodle.png'))

But that doesn't do me a whole lot of good since I just want to get the csv and parse it rather than write it.

Solution

  • According to its documentation, csvtojson supports streams (there are examples of it in the docs). So you would just do something like:

    var request = require('request');
var Converter = require('csvtojson').Converter;

request('http://example.org/foo.csv')
  .pipe(new Converter({constructResult:true}))
  .on('end_parsed', function(jsonObj) {
    console.dir(jsonObj);
  });