Programmatically Obtain All YouTube Videos Belonging to a User

Question

Utilizing the YouTube Data API we can make a query to to obtain the first 50 (maximum amount of results obtainable with a single query) videos belonging to a user with the following request: https://www.googleapis.com/youtube/v3/search?key={access_key}&channelId={users_id}&part=id&order=date&maxResults=50&type=video

If there are more than 50 videos, then the resulting JSON will have a nextPageToken field, to obtain the next 50 videos we can append &pageToken={nextPageToken} to the above request, producing the next page of 50 videos. This is repeatable until the nextPageToken field is no longer present.

Here is a simple JavaScript function I wrote using the fetch API to obtain a single page of videos, specified by the nextPageToken parameter (or lack thereof).

function getUploadedVideosPage(nextPageToken) {
    return new Promise((resolve, reject) => {
        let apiUrl = 'https://www.googleapis.com/youtube/v3/search?key={access_key}&channelId={users_id}&part=id&order=date&maxResults=50&type=video';
        if(nextPageToken)
            apiUrl += '&pageToken=' + nextPageToken;

        fetch(apiUrl)
        .then((response) => {
            response.json()
            .then((data) => {
                resolve(data);
            });
        });
    });
}

Now we need a wrapper function that will iteratively call getUploadedVideosPage for as long as we have a nextPageToken. Here is my 'working' albeit dangerous (more on this later) implementation.

function getAllUploadedVideos() {
    return new Promise((resolve, reject) => {
        let dataJoined = [];
        let chain = getUploadedVideosPage();

        for(let i = 0; i < 20000; ++i) {
            chain = chain
            .then((data) => {
                dataJoined.push(data);

                if(data.nextPageToken !== undefined)
                    return getUploadedVideosPage(data.nextPageToken);
                else
                    resolve(dataJoined);
            });
        }
    });
}

The 'dangerous' aspect is the condition of the for loop, theoretically it should be infinite for(;;) since we have no predefined way of knowing exactly how many iterations to make, and the only way to terminate the loop should be with the resolve statement. Yet when I implement it this way it truly is infinite and never terminates.

Hence why I hard coded 20,000 iterations, and it seems to work but I don't trust the reliability of this solution. I was hopping somebody here can shed some light on how to go about implementing this iterative Promise chain that has no predefined terminating condition.

Answer 1

You can do this all with one function that calls itself if applicable.

You are also using an explicit promise construction anti-pattern wrapping fetch() in new Promise since fetch() already returns a promise

function getVideos(nextPageToken, results = []) {

  let apiUrl = 'https://www.googleapis.com/youtube/v3/search?key={access_key}&channelId={users_id}&part=id&order=date&maxResults=50&type=video';
  if (nextPageToken) {
    apiUrl += '&pageToken=' + nextPageToken;
  }

  // return fetch() promise
  return fetch(apiUrl)
    .then(response => response.json())
    .then(data => {
      // merge new data into final results array
      results = results.concat(data);

      if (data.nextPageToken !== undefined) {
        // return another request promise
        return getVideos(data.nextPageToken, results);
      } else {
        // all done so return the final results
        return results
      }
    });
}

// usage
getVideos().then(results=>{/*do something with all results*/})
           .catch(err=>console.log('One of the requests failed'));