Node arrays asynchronous error

This might be an error outside of asynchronous calls, in which case a fresh pair of eyes is probably good at any rate.

I'm writing a function that interacts with the flickr api. Given a user, it grabs their ID, uses their ID to get a list of sets (albums) of the user, and given that list of sets, gets the list of photos in each set.

Here is the code that calls the problem function (just for context; this all works afaik):

router.post('/getPhotosFromSets', function(req, res) {
var uname = req.body.Username;
getIdFromUsername(uname, function(id) {
    console.log("ID Received: " + id);
    getSetsFromId(id, function(setList) {
        console.log("setList received: " + JSON.stringify(setList));
        getURLsFromSets(setList, function(){
            console.log("\n\nSetlist received: "+JSON.stringify(setList));
            res.render('userPhotos', {'setList': setList});
        });
    });
});

I know that the nested async calls aren't best practice, but there are only 3 so it's manageable and I don't want to have to learn async.js unless I really have to.

I know that getID... and getSets... work. The problem is in getURLsFromSets. Basically what I want to do is, I get a list of lists from getSetsFromId. Like such:

[[Album Name, Album ID],[Album2 Name, Album 2 ID]].

To each of these lists, I want to attach a list of photo URLs. So it would look like:

[[Album Name, Album ID, [Photo1 URL, Photo2 URL]],[Album2 Name, Album 2 ID, [Photo1 URL]].

Here is the code for getURLSFromSets:

getURLsFromSets = function(setList, callback) {
console.log("setList argument: "+setList);
var activeThreads = setList.length;
var onComplete = function() {
    callback(setList);
};

for(var i = 0; i < setList.length; i++) {
    var setInfo = setList[i];

    // URL is query to grab photos from a set
    var url = 'https://api.flickr.com/services/rest/?';
    url += 'method=flickr.photosets.getPhotos&api_key=';
    url += 'MY_API_KEY'; // API Key
    url += '&photoset_id='+setInfo[1];
    url += '&format=json&nojsoncallback=1';
    console.log("Set info before request: " + setInfo);

    request(url, function(error, response, body) {
        console.log("Set info at request: " + setInfo);
        var photos = JSON.parse(body).photoset.photo;
        setInfo.push(stringifyUrl(photos));
        if (--activeThreads === 0) {
            onComplete();
        }
    });
}
};

setList, the argument taken in, is that list of lists I talked about above. So it wants to go through each list in the setList of lists, use the info in the list to get a URLlist, and append the URLlist to the list. Sorry if that is unclear. But the end result of what I want is a list of lists, where each sublist contains the album info and a list of photos.

Now, this is where the problem arises. When I'm doing this for a user that has just 1 album, this method works perfectly. For example, when I enter my own ID, djbhindi, I get this:

Setlist received: [["Desktops","72157645413282948",["https://farm6.staticflickr.com/5233/14669675871_4882577de4_b.jpg","https://farm4.staticflickr.com/3861/14486214499_b492c8c6c3_b.jpg"...]]]

It is a list which has only 1 sublist, and that sublist represents 1 album with the name "desktops", the id "721576..." and a list of photo URLs as required.

However, when I enter a user that has more than 1 album (I found a random one named 'joshuamalik') this gets screwed up:

Setlist received: [["365","72157630946802318"],["Images.","72157629614042425",["https://farm8.staticflickr.com/7106/7730306384_e07f5cff20_b.jpg",...],["https://farm8.staticflickr.com/7063/6994331163_e41e9687db_b.jpg","https://farm8.staticflickr.com/7108/7019359683_28dc6dc2e6_b.jpg"]]]

This might be unclear, but basically the guy has 2 albums, one with 2 photos and one with 14. What the method does is return a list of lists that contains 2 sublists, as desired (1 representing each album), but the list of photo URLs for the first album gets added to the second album's list, and then the second album's list gets added as well.

So I ended up with:

[[Album 1 info] [ Album 2 info, [Album 1 photos], [Album 2 photos]]]

Instead of

[[Album 1 info, [Album 1 photos]], [Album 2 info, [Album 2 Photos]]]

My theory is that the async call in getUrsFromSets has setInfo already bound to the Album 2 sublist by the time the call completes, so it appends to the wrong thing.

Any ideas? Thank you guys so much for reading this whole thing. I know it's not an authorization issue because otherwise I wouldn't be getting back photo URLs from the other guy. It's just a formatting issue.

Solution

Closure problem. Basically setInfo becomes a reference to the last iterated setList element. Wrapping the ajax callback in an immediate function that takes the correct setInfo as an argument is one way of fixing it. Try making these changes.

request(url, function(error, response, body) {
    console.log("Set info at request: " + setInfo);
    var photos = JSON.parse(body).photoset.photo;
    setInfo.push(stringifyUrl(photos));
    if (--activeThreads === 0) {
        onComplete();
    }
});

becomes

request(url, 
  (function (setInfo) {
    return function(error, response, body) {
      console.log("Set info at request: " + setInfo);
      var photos = JSON.parse(body).photoset.photo;
      setInfo.push(stringifyUrl(photos));
      if (--activeThreads === 0) {
        onComplete();
      }
    }
  })(setInfo)
);