Run Mongo find Synchronously

I have a problem where I've got 20+k rows in a csv file and I'm trying to update them based on documents of a matching field in a Mongo DB that contains 350k docs.

The trick is that I need to perform some logic on the matches and then re-update the csv.

I'm using PapaParse to parse/unparse the csv file

Doing something like works to get all my matches

const file = fs.createReadStream('INFO.csv');
Papa.parse(file, {
    header: true,
    complete: function(row) {        
        getMatchesAndSave(row.data.map(el => { return el.fieldToMatchOn })); 
    }
});`

function getMatchesAndSave(fields) {
Order.find({fieldToMatchOn: { $in: fields}}, (err, results) => {
    if (err) return console.error(err);
    console.log(results);
});

}

That gets me matches fast. However, I can't really merge my data back into the csv bc the csv has a unique key column that Mongo has no idea about.

So all the data is really dependent of what's in the csv.

Therefore I thought of doing something like this

const jsonToCSV = [];
for (let row of csvRows) {
    db.Collection.find({fieldToMatchOn: row.fieldToMatchOn}, (err, result) => {
        //Add extra data to row based on result
        row.foo = result.foo;

        //push to final output
        jsonToCSV.push(row);
    }
}
papa.unparse(jsonToCSV);
//save csv to file

The issue with the above implementation (as terribly inefficient as it may seem) - is that the Find calls are asynchronous and nothing gets pushed to jsonToCSV.

Any tips? Solving this with $in would be ideal, are there any ways to access the current element in the $in (so looking for the iterator)..that way I could process on that.

Solution

You can try async/await to iterate csvRows array, like this:

const search = async () => {
  const jsonToCSV = await Promise.all(csvRows.map(async row => {
/* find returns a promise, so we can use await, but to use await is 
mandatory use it inside an async function. Map function not returns a 
promise, so this can be solve using Promise.all. */
    try {
      const result = await db.Collection.find({ fieldToMatchOn: row.fieldToMatchOn });
      row.foo = result.foo;
      return row;
    } catch (e) {
      // do somenthing if error
    }
  }));
  papa.unparse(jsonToCSV);
}

// call search function
search();

Check this https://flaviocopes.com/javascript-async-await-array-map to a better understanding.