I have an ExpressJS app which takes form data and does the following:
I'm using mongoskin for the database.
My question relates to how I control the flow. Essentially I have written each of the above steps as a middleware function because I need to call next() on the success (or next(err) on error) at each callback.
It seems like I'm writing too much middleware and should be able to group the steps into larger sets of middleware containing multiple 'sub-functions' but I'm not sure how to do this in Express since I need to call next() every time an async function call completes. Is there a correct way to do this or is this 'one middleware per step' approach really the right way to run this?
EDIT: Posting some code as requested. This is partial code for the sake of brevity:
function validateFields(req, res, next) {
//...
//iterate over req.body to confirm all fields provided
//...
if (allDataProvided) {
//...
//iterate over req.body to confirm all fields valid
//...
if (allDataValid) {
return(next());
} else {
return(next(err));
}
} else {
return(next(err));
}
},
//get an auto incrementing ID fields from a mongodb collection (counters)
function getNextID(req, res, next) {
counters.findAndModify(
{ _id: "receiptid" },
[['_id','asc']],
{ $inc: { seq: 1 } },
{},
function(err, doc) {
if (err) {
return next(err);
} else {
req.receiptid = doc.seq;
return next();
}
});
},
//insert a new record into the transaction collection (txns) using the new ID
function createTransaction(req, res, next) {
txns.insert(
{ _id : req.receiptid,
body : req.body,
status : "pending"},
{},
function(err, r) {
if (err) {
return next(err);
} else {
return next();
}
});
},
//process the data on the remote web service using the provider's API (remoteapi)
function processTransaction(req, res, next) {
remoteapi.processTransaction(
{ data: req.body,
receiptid: req.receiptid },
function(err, r) {
if (err) {
return next(err);
} else {
req.txnReceipt = r;
return next();
}
});
},
//update the record in the database collection (txns) with the server response
function updateDatabase(req, res, next) {
txns.updateById(req.receiptid,
{ $set :{status : "success",
receipt: req.txnReceipt }
}, function (err, r) {
if (err) {
return next(err);
} else {
return next();
}
});
}
And as it currently stands with the above functions, my route which utilises this middleware starts like this:
router.post('/doTransaction',
validateFields,
getNextID,
createTransaction,
processTransaction,
updateDatabase,
function(req, res, next) { //...
It seems like I should be able to create one middleware function which does all of these things in a row without each having to be a separate middleware, but since each middleware has an async function in it and I need to call next() in the resulting callback, this is the only way I can see it working.
Thanks Nicolai and robertklep for the answers. Whilst I think both answers do answer the question, I realised as I was working through this myself that I had failed to see the forest for the trees.
I could just pass the next function through each callback function until I reached the final one and call it to pass the control back to the middleware stack. This also allows me to simply call next(err) inside any of those functions.
So my answer is very similar to the concept outlined by Nicolai except I don't think I need to use the async package in this case because I don't feel like this particular case took me to callback hell.
Here is my answer to my own question:
function validateFields(req, res, next) {
//...
//iterate over req.body to confirm all fields provided
//...
if (allDataProvided) {
//...
//iterate over req.body to confirm all fields valid
//...
if (allDataValid) {
getNextID(req, res, next)
} else {
return(next(err));
}
} else {
return(next(err));
}
},
//get an auto incrementing ID fields from a mongodb collection (counters)
function getNextID(req, res, next) {
counters.findAndModify(
{ _id: "receiptid" },
[['_id','asc']],
{ $inc: { seq: 1 } },
{},
function(err, doc) {
if (err) {
return next(err);
} else {
req.receiptid = doc.seq;
createTransaction(req, res, next);
}
});
},
//insert a new record into the transaction collection (txns) using the new ID
function createTransaction(req, res, next) {
txns.insert(
{ _id : req.receiptid,
body : req.body,
status : "pending"},
{},
function(err, r) {
if (err) {
return next(err);
} else {
processTransaction(req, res, next);
}
});
},
//process the data on the remote web service using the provider's API (remoteapi)
function processTransaction(req, res, next) {
remoteapi.processTransaction(
{ data: req.body,
receiptid: req.receiptid },
function(err, r) {
if (err) {
return next(err);
} else {
req.txnReceipt = r;
updateDatabase(req, res, next);
}
});
},
//update the record in the database collection (txns) with the server response
function updateDatabase(req, res, next) {
txns.updateById(req.receiptid,
{ $set :{status : "success",
receipt: req.txnReceipt }
}, function (err, r) {
if (err) {
return next(err);
} else {
return next();
}
});
}
So instead of calling next() on successful completion of each async function and having to write another middleware for the next step, I simply pass next on to the next function until it's required.
This was, I can just call the first function as my middleware, like this:
router.post('/doTransaction',
validateFields,
function(req, res, next) { //...
and in turn, the remaining steps are called in sequence when each action completes.