How to handle long paths using std::filesystem in c++17

Question

On Windows I am trying to iterate and collect all the files in the directory, Also I want to handle long paths and thats why I've added long path prefix (L"\\?\"). but it crashes with c++17's std::filesystem. Also I've tried the same with boost::filesystem and It also crashed, However If I don't add the prefix, it works. I want this code to work on mostly on any version of windows-10/11. How do I handle the possibility of encountering long path.

    void IterateDir(const std::string& newDir)
    {
        namespace fs = std::filesystem;
    
        fs::path rootPath;
        std::wstring wpath(newDir.begin(), newDir.end());
        wpath = std::wstring(L"\\\\?\\") + wpath;
        rootPath.assign(wpath);
    
        for (auto iter = fs::recursive_directory_iterator(rootPath);
            iter != fs::recursive_directory_iterator();
            ++iter) {
            try {
                auto file = *iter;
                std::string filename = file.path().filename().string();
                std::cout << "\nfile: " << filename;
            }
            catch (const std::exception& e) {
                std::cout << "Exception while iterating directory." + std::string(e.what());
            }
            catch (...) {
                std::cout << "Unknown exception while iterating directory.";
            }
        }
    }

Thanks in advance :)

What I've found that, There is a known issue with the C++ 17 filesystem crashing when iterating a directory with a long path prefix in wstring. The issue is caused by the way the filesystem::recursive_directory_iterator handles the files. When the iterator reaches a file with Unicode characters, it crashes.

Answer 1

The Object Manager Paths will not work with either std::filesystem or boost::filesystem (you need platform specific API like NtOpenDirectoryObject, NtQueryDirectoryObject).

That said, neither library ought to crash. Barring bugs (which are unlikely, and if they bite you, you can probably upgrade the library to fix it), you might simply be running into permission errors which will cause runtime exceptions to be raised.

So, here's a version that simplifies the code while adding minimal precautions to deal with permission problems during traversal:

Live On Coliru

#include <filesystem>
#include <iostream>
namespace fs = std::filesystem;

void IterateDir(fs::path dir, char const* nsPrefix = R"(\\?\)") {
    auto flags = fs::directory_options::skip_permission_denied;
    for (auto&& entry : fs::recursive_directory_iterator(nsPrefix / dir, flags)) {
        try {
            std::cout << "file: " << entry.path().filename() << "\n";
        } catch (std::exception const& e) {
            std::cout << "Exception while iterating directory." + std::string(e.what()) << "\n";
        } catch (...) {
            std::cout << "Unknown exception while iterating directory.\n";
        }
    }
}

int main() { 
    IterateDir(".", ""); 
    IterateDir("."); 
}

You can see the first iterate works:

file: "main.cpp"
file: "a.out"

The second one raises an exception (that goes unhandled, leading to program termination):

terminate called after throwing an instance of 'std::filesystem::__cxx11::filesystem_error'
  what():  filesystem error: recursive directory iterator cannot open directory: No such file or directory [\\?\/.]

On windows you might get different behaviour, but a crash should never happen.

BONUS

Boost version Live On Coliru

#include <boost/filesystem.hpp>
namespace fs = boost::filesystem;

showing the same output except the exception is:

terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
  what():  boost::filesystem::directory_iterator::construct: No such file or directory [system:2]: "\\?\/."