Why is my Node.js function (which involves fs.readdir) executed after the subsequent code fragment?

I came across a problem when trying to read a directory with Node.js (v8.9.4) in Visual Studio Code. I have a function called readExperiments which runs trough my directory and returns an array of experiments. But somehow the code which comes after the function call is executed first (which results in an "undefined" problem scenario). The console.log statements inside experimentReader.js and the for loop in mustacheModelCreator.js are there for debugging purposes.

Is there a trivial reason why my function is executed afterwards (maybe due to its horrible nesting style) and is there a solution which forces my readExperiments() function to be executed first?

I thank you very much for your help (btw: I am a total javascript beginner).

Code in mustacheModelCreator.js

var experiments = experimentReader.readExperiments();
for(k = 0; k < 1;k++)
{
    console.log("Outside experiments: " + experiments[k]);
}
var experimentPaths = experimentReader.getExperimentPaths();
var patients = patientReader.readPatients(experimentPaths);

Code in experimentReader.js:

module.exports.readExperiments = function()
{
    var experiments = new Array();
    fs.readdir(rootPath, function(err, rootDirectory)
    {
        if(err) throw err;
        for(i = 0; i < rootDirectory.length; i++)
        {
            console.log("Root Directory: " + rootDirectory[i]);
            if(rootDirectory[i]=="resources")
            {
                var resPath = path.join(rootPath,'/resources');
                fs.readdir(resPath, function(err2, resDirectory)
                {
                    console.log("Resources Directory: " + resDirectory[i]);
                    if(err2) throw err2;
                    for(j = 0; j < resDirectory.length; j++)
                    {
                        if(resDirectory[j] == "results")
                        {
                            var resultsPath = path.join(resPath,'/results');
                            var resultsDirectory;
                            fs.readdir(resultsPath, function(err3, resultsDirectory)
                            {
                                console.log("Results Directory: " + resultsDirectory[i]);
                                if(err3) throw err3;
                                for(p = 0; p<resultsDirectory.length;p++)
                                {
                                    experiments.push(resultsDirectory[p]);
                                    console.log("Inside experiments: " + experiments[p]);
                                }
                            });
                        }
                    }
                });
            }
        }
    });
return experiments;
}

Result:

Outside experiments: undefined

Then the internal console.log statements from readExperiments in experimentReader.js

Solution

Your problem lies in the fact that your are using asynchronous functions.

when you say:

fs.readdir(path, function callback(x){} );

your callback function will be executed at some later point in time, it may be right after the fs.readdir invocation, or it may be an hour later (usually a few milliseconds though :))

try replacing your usage of readdir with readdirSync, see documentation on usage:

https://nodejs.org/api/fs.html#fs_fs_readdirsync_path_options

Another alternative would be to use Promises or just plain old callbacks and embrace asynchronous nature of node.js

So to sum it up:

you call experimentReader.readExperiments();
this starts the process of directory reading
execution continues onto

for(k = 0; k < 1;k++) { console.log("Outside experiments: " + experiments[k]); }
meanwhile code inside your readExperiements runs, and it so happens that most of the time it finishes after your loop that looks at experiments array.