Node JS: How do I request multiple source codes from different websites?

Question

I get an issue where If I try to use request more than once, it doesn't work. I can give an example:

request('https://www.google.com', function (error, response, vBody1) {
    request('https://www.purple.com', function (error, response, vBody2) {
        request('https://www.stackoverflow.com', function (error, response, vBody3) {
            console.log(vBody3);
        });
        console.log(vBody2);
    });
    console.log(vBody1);
});

It reads and outputs the body of the first site, but not the next two. Here is another example:

vSites = ['https://www.google.com','https://www.purple.com','https://www.stackoverflow.com'];
for (i = 0; i < vSites.length; i++){
    request(vSites[i], function (error, response, vBody[i]) {
        console.log(vBody[i]);
    });
}

This one looks cleaner and I would like this version, but it doesn't work at all. What am I doing wrong?

Answer 1

In addition to the correct solution from Michael's reply, please be aware the for-loop sends the requests in parallel. Data and error handling could be challenging if you want to do more than console.log.

To better handle the requests, I would suggest using an asynchronous control library such as async or bluebird. In comparison, async is easier to understand while bluebird provides better code readability.

Below is an example async implementation:

var async = require('async');
var request = require('request');

var vSites = ['https://www.google.com', 'http://www.purple.com','https://www.stackoverflow.com'];
async.map(
  vSites,
  // `async` will call this function for each `vSite`
  function(vSite, next) {
    // Reuse the code inside your for-loop, but call `next` to pass the error and result to the final callback
    request(vSite, function(err, response, vBody) {
      next(err, vBody);
    });
  },
  // The final callback will be executed when all requests are completed or either of them fails
  function(err, results) {
    console.log(err);
    console.log(results);
  }
);

Hope this helps :)