How to use "async/await" and "promises" for synchromous execution in Node.js?

Question

I am fairly new to web development as a whole, so I don't understand promises and async/await much. I'm trying to build a job posting site as a project, but got stuck as I'm unable to make a particular piece of code behave as I want.

In the following snippet, the recruiter who posted a particular job requirement wants to see the skills of the applicant that are relevant to the job requirement. So the code would first get the job details and the details of the applicant in question from the mongodb collection. Then it'll try to first store the required skill of the applicant and then all the skills of the applicant in the skills array. Only after storing all of the above data in skills[], should the code should respond with the last json. But it's not behaving like I want and its responding before the storing is complete. Can someone guide me on how to make it "synchronous"?

NOTE: I removed all the error handling to make the snippet shorter. Also, the applicant has 2 required skills out of overall 5 skills.

router.post('/recruiter/viewApplicant', passport.authenticate('jwt', {session: false}), (req, res, next) => {

    if(req.user.accountType === "Recruiter") {
        skills = [];

        Job.getJobByID((req.body.jobID),(err, job) => {
            User.getUserByEmail(req.body.email,(err, user) => {
                let i =0;
                job.requiredSkills.requirements.forEach(reqSkill => {
                    user.classData.forEach(classDetail => {
                        if(classDetail.classID === reqSkill.class.replace(/ /g, '-')) {
                            ClassData.getClassByclassID(classDetail.classID,(err, classInfo) => {
                                skills.push({
                                    type: 'req',
                                    imgURL: classInfo.imgURL,
                                    skill: classDetail
                                })
                                console.log('req: '+i)
                                i++
                            })
                        }
                    })
                })

                let k=0
                user.classData.forEach(classDetail => {
                    ClassData.getClassByclassID(classDetail.classID,(err, classInfo) => {
                        skills.push({
                            type: 'all',
                            imgURL: classInfo.imgURL,
                            skill: classDetail
                        })
                        console.log('all: '+k)
                        k++
                    })
                })
            }) 

            console.log("Completed");        
            res.json({success: true, msg: "Successful", data: skills});
        })
    }
});

Expected Result:

req: 0  
req: 1   
all: 0  
all: 1  
all: 2  
all: 3  
all: 4  
Completed  

Actual Result:

Completed  
req: 0  
req: 1  
all: 0  
all: 1  
all: 2  
all: 3  
all: 4  

Answer 1

To get the code you have to work, move the res.json() call as follows

            user.classData.forEach(classDetail => {
                ClassData.getClassByclassID(classDetail.classID,(err, classInfo) => {
                    skills.push({
                        type: 'all',
                        imgURL: classInfo.imgURL,
                        skill: classDetail
                    })
                    console.log('all: '+k);
                    k++;
                })
            })
            res.json({success: true, msg: "Successful", data: skills});

It doesn't work where you put it because your call to .getJobById() returns immediately. When it finishes the work you want from it, you know because it invokes its callback function. The same is true of the nested .getUserByEmail() call.

async/await and Promise are ways to organize Javascript callbacks so they do not appear to be nested. They make code easier to read. But the callbacks still are nested and asynchronous. Explaining exactly how to convert your code to use Promises and async/await is beyond the scope of a SO answer. In short, you should promisify your asynchronous functions, then you can await their results. It is absolutely worth a day or two of your time to get familiar with this way of coding in Javascript.

Pro tip: If you step through this kind of code with your debugger, use Step Into a lot.