Javascript Promise.all() not resolving or rejecting

Question

Given the following code:

                let promiseArray = [];
fs.readdir('someFolderHere', (err, fileList) => {
                filesList.forEach(filename => {
                    // readFile returns a promise that resolves 
                    // to an array of strings to insert into the database
                    readFile(fileName).then((records) => { 
                        records.forEach((record) => {
                            promiseArray.push(db.insert(record)); // db.insert returns a promise
                        });
                    });
                });

                // console.log(promiseArray); // empty array here??? 
    
                Promise.all(promiseArray).then((res) => {
                    console.log('resolved');
                    resolve(res);
                }).catch((err) => {
                    console.log('rejected');
                    reject(err);
                });
    });

I cannot figure out why the promiseArray isn't resolving in the Promise.all(). Neither of those console.log statements print out.

At a guess, I would suppose that the Promise.all is called before the promiseArray is populated, so that it is a blank array, e.g. Promise.all([ ]).then(...)

How do I get around this. I am trying to read a list of files, for each file, I am reading several lines of data. I need all records from all files to be inserted into the database - promises just are not playing nice. Is there any way to make the Promise.all wait until promiseArray has been fully populated and the forEach is fully completed?

Answer 1

Once you stop using then, your code will automagically get shorter and cleaner. Example:

let readFile = path => [path + ' record1', path + ' record2']
let insert = record => 'insert ' + record

async function test(fileList) {
    let records = await Promise.all(fileList.map(path => readFile(path)))
    return Promise.all(records.flat().map(rec => insert(rec)))
}

test(['A', 'B', 'C']).then(console.log);

These two lines do the same as your 10+ lines function.

In response to your comment, you can apply the same generic pattern when working with "exploding" maps, where each element maps to an array (like folders -> files in each folder, files -> records in each file etc).

let folders = await Promise.all(serverList.map(server => getFolderOn(server)))
let files = await Promise.all(folders.flat().map(folder => getFilesIn(folder)))
let records = await Promise.all(files.flat().map(file => getRecordsIn(file)))

and so on...

You can save a bit of typing, and define a generic function like

let flatPromise = (arrayOfArrays, mapper) =>
    Promise.all(arrayOfArrays.flat().map(mapper))

and then

let folders = await flatPromise(serverList, server => getFolderOn(server))
let files = await flatPromise(folders, folder => getFilesIn(folder))
let records = await flatPromise(files, file => getRecordsIn(file))
....