What algorithm can I use to recursively load an entire directory, starting from a specified path?

Question

I'm building a custom file dialog, and my problem is its taking too long to load.

The dialog begins with an InitialDirectory property, and I am looking to find a way to load the directory tree for the InitialDirectory first, followed by the rest of the directories in a background thread.

For example, if the InitialDirectory was C:\Users\User12345\MyDocuments, then it should load all folders in

C:\
C:\Users
C:\User12345
C:\Users\User12345\MyDocuments

then kick off a background thread to load all the remaining directories.

Is there an fast and easy way to use recursion to load first the InitialDirectory, than everything else, without duplicating any items?

I'm struggling to find a high-performing way to do this, since checking for the existance of a folder with code like if (!Directory.Contains(f => f.FullName == folder.FullName)) slows down the load by quite a bit.

My current code to load the full directory looks something like this:

private void LoadDirectory()
{
    string root = @"C:\";
    var rootNode = new DirectoryModel() { Name = root, FullName = root };
    this.Directory.Add(rootNode);

    DirectoryInfo info = new DirectoryInfo(root);
    IEnumerable<DirectoryInfo> subDirectories = info.GetDirectories()
            .Where(d => ((d.Attributes & FileAttributes.Hidden) != FileAttributes.Hidden)
                && ((d.Attributes & FileAttributes.System) != FileAttributes.System));

    LoadDirectories(subDirectories, root);
}

private void LoadDirectories(IEnumerable<DirectoryInfo> subDirs, string parentName)
{
    IEnumerable<DirectoryInfo> subdirectories;
    foreach (DirectoryInfo folder in subDirs)
    {
        var node = new DirectoryModel() { Name = folder.Name, FullName = folder.FullName, ParentName = parentName };

        Directory.Add(node);

        try
        {
            subdirectories = folder.GetDirectories("*", SearchOption.TopDirectoryOnly)
                .Where(d => ((d.Attributes & FileAttributes.Hidden) != FileAttributes.Hidden)
                    && ((d.Attributes & FileAttributes.System) != FileAttributes.System));
        }
        catch (UnauthorizedAccessException e)
        {
            continue;
        }
        catch (System.IO.DirectoryNotFoundException e)
        {
            continue;
        }

        if (subdirectories.Count() != 0)
            LoadDirectories(subdirectories, folder.FullName);
    }
}

Note that the Directory collection is a flat collection - there is no hierarchy in the data models.

Answer 1

It depends on how you present to the user your data, imo.

If you use a TreeView control, you can easily avoid to load all data in background, but just load only the root of your tree (like you already do as much as I understood).

In this way, having a procedure that loads all sub-directory names and files of a specified directory you just need to wait until user clicks on the [+] in order to see the directory content interesting to him. So you run your procedure and populate the tree.

This is actually highest performant way known to me to deal with this kind of stuff, and actually you will find this pattern almost in all "explorer like" products. The performance here achieved not by perfectly refined algorithm, but just by defining more convenient behavioral model.

Hope this helps.