Search code examples
javascriptnode.jses6-promisebluebirdknex.js

Using each promise value of Promise.map as input of the next loop


I've done whole-day research on how I can get result of each promise in a Promise.map and use it as input in the next iteration during loop of that same Promise.map. I strictly need to do this approach because am using this logic in a database operation that must be ATOMIC and in case of any promise rejection all the previous transactions have to be rolled back.

NOTE: I have used promise.each and it works well only that it does not allow me to associate the individual promises and roll back all if one fails. So Promise.map seem to be the best solution when each promise is carefully resolved and value returned without causing Error: Transaction query already complete in the next loop. Here is the logic with knex:

var obj={};

knex.transaction(function(trx) {
  return Promise.map(array, function(item) {
        return trx.insert(item).into('table')
        .then(returnedFields => {
        //do some validation/operation and return the result or the returnedFields themselves as input in the next loop.
        //[START EDIT: this responds to comment by @ Mikael Lepistö for clarity]
        //update obj here to be used in next loop
        //[END EDIT]
      });
    }, {concurrency: 1});
})
.then(function(inserts) {
  console.log(inserts.length + 'Items saved.');
})
.catch(function(error) {
  console.error(error);
})


Solution

  • Thank you everyone who posted great/greater insights. However it has turned out that Promise.each works well with knex and I was making a mistake of calling commit inside then of some individual promises during the loop hence causing Error: Transaction query already complete in next transaction attempt of the subsequent loop. Reason: There was no need calling commit withing the knex transaction context/block since it's automatically triggered in that context.

    To note in the answer below:

    Using Knex with Promise.each requires you to listen to possible rejection inside thenblock of each promise and with a try/catch and in some cases explicitly reject else the subsequent promises/values will continue looping and that cannot make database atomic!!!

    knex.transaction(function(trx) {
      return Promise.map(array, function(item) {
            return trx.insert(item).into('table')
            .then(returnedFields => {
            try{
            //do some validation/operation and return the result or the returnedFields themselves as input in the next loop.
            /* START Testing a case of rejection */
               if (array.indexOf(item) === 3) {
                   //uncomment the code below to test
                   //throw new Error('BreakException');
                }
            /* END */
            }catch(err){
            fail=true;
            }
          }).then(val => {
              if (fail) {
                trx.rollback()//you can ignore this as is automatically triggered here - tested
                return Promise.reject(new Error('Rejected'))
              }
              return val
            })
            .catch(err => {
              fail = true
              trx.rollback()//you can ignore this as is automatically triggered here - tested
              return Promise.reject(new Error('Rejected'))
            });
        });
    })
    .then(function(inserts) {
      console.log(inserts.length + 'Items saved.');
    })
    .catch(function(error) {
      console.error(error);
    })