Search code examples
javascriptes6-promise

Promise.all() Combined with setTimeout to Throttle API Calls


I have a set of n users where I need to hit an API endpoint for each user to get more information, which is quite easy with Promise.all(), but now my n has gotten to be much bigger and I now need to throttle my calls to about 50 per 5 seconds. Is there a way using .reduce() combined with setTimeout(() => {}, 5000) to batch 50 calls every 5 seconds, but still capture the results of every call in Promise.all().then()?

// Retrieve data from an API endpoint for n users [works]
const users = [...array of 50 ids];

Promise.all(users.map(user => axios.get(`api/users/${user.id})))
       .then(results => resolve())
       .catch(err => reject(err));

Solution

  • you could do something like this, this is slightly pseudo, but i am sure you get the gist of it :):

    const batchSize = 50;
    const promises = [];
    const users = [...lots of users];
    const userCount =  users.length;
    let userIdx=0;
    
    function getUsers() { 
       for(let i=0;i<batchSize;i++) {
          if(userCount < ++userIdx) {
            Promise.all(promises).then(doYourStuff);
            return;
         }
         promises.push(axios.get('api/users/' + users[userIdx]));
       }       
       setTimeout(_ => getUsers(), 5000);
    }