Nested async await not working as expected

Question

I'm trying to add a table in my existing code for some data and for this i have setted up everything similar to what mentioned in the exceljs docs. Here is a code snippet of the condition where i'm trying to get the data i need in order to get the table. When i'm printing the modRows within the forEach, it's showing the data but when i'm printing it outside the loop it's getting blank. Is there any way or other workaround to this? Is it possible to have async within async?

const generateCatReports = async (mode = 'monthly', months = 1, toDate = false) => {
  if (!['monthly', 'weekly'].includes(mode)) {
    throw new Error(InvalidParamsError);
  }
 const sortedTRIds = obtainSortedCentreIds(studiesTRMap, centreNameIDMap);
  const modRows = [];
  sortedTRIds.forEach(async (trId) => {
    console.log("trid",trId);
    const statsParams = {
      catId: trId,
      createdAt,
    };
    const statsPipeline = studiesStatsPipeline(statsParams, capitalize(mode));
    const statsInfo = await StudyModel.aggregate(statsPipeline).allowDiskUse(true).exec();
    Object.entries(statsInfo[0]).slice(1).forEach(([key, val]) => {
      modRows.push([key, val]);
    });
    console.log("inside sortedTr loop>>>>>" ,modRows);
  });
  console.log("outside sortedTr loop>>>>>>>>",modRows);

}

Result:

trId 1cf1eb1324322bbebe
inside sortedTr loop>>>>>>>>>>
['TOTAL', 10]
['white', 5]
['black', 5]

trId 1cf1eb1324322bbebe
inside sortedTr loop>>>>>>>>>>
['TOTAL', 10]
['white', 5]
['black', 5]

trId 21e1eb21322bbebe
inside sortedTr loop>>>>>>>>>>
['TOTAL', 8]
['white', 6]
['black', 2]

outside sortedTr loop>>>>>>>>>>
[]

Answer 1

Whenever you see an async callback like .forEach(async (trId) => { the chance that something is wrong is pretty high.

The problem what you have is that async functions are actually promises and therefore they don't block the main thread. That means that your callback function gets queued in the job queue and will be executed in the future.

Its simplified the same like this:

let arr = []
setTimeout(() => {
   arr.push("hy")
})
console.log(arr)

arr will be simply empty,

However you can use an for ... of loop

const generateCatReports = async (
  mode = "monthly",
  months = 1,
  toDate = false
) => {
  if (!["monthly", "weekly"].includes(mode)) {
    throw new Error(InvalidParamsError);
  }
  const sortedTRIds = obtainSortedCentreIds(studiesTRMap, centreNameIDMap);
  const modRows = [];
  for (const trId of sortedTRIds) {
    const statsParams = {
      catId: trId,
      createdAt
    };
    const statsPipeline = studiesStatsPipeline(statsParams, capitalize(mode));
    const statsInfo = await StudyModel.aggregate(statsPipeline)
      .allowDiskUse(true)
      .exec();
    Object.entries(statsInfo[0])
      .slice(1)
      .forEach(([key, val]) => {
        modRows.push([key, val]);
      });
    console.log("inside sortedTr loop>>>>>", modRows);
  }
  console.log("outside sortedTr loop>>>>>>>>", modRows);
};

Here you have no callback that will be queued.

A better solution would be to use Promise.all()

const generateCatReports = async (
  mode = "monthly",
  months = 1,
  toDate = false
) => {
  if (!["monthly", "weekly"].includes(mode)) {
    throw new Error(InvalidParamsError);
  }
  const sortedTRIds = obtainSortedCentreIds(studiesTRMap, centreNameIDMap);
  const modRows = await Promise.all(
    sortedTRIds.flatMap(async (trId) => {
      const statsParams = {
        catId: trId,
        createdAt
      };
      const statsPipeline = studiesStatsPipeline(statsParams, capitalize(mode));
      const statsInfo = await StudyModel.aggregate(statsPipeline)
        .allowDiskUse(true)
        .exec();
      return Object.entries(statsInfo[0]).slice(1);
    })
  );

  console.log(modRows);
};