I am writing a linux command line program that will return the size of a directory. The program works as expected, except when specifically dealing with root directories. I know many files in the root directory do not have sizes because they are special files used to represent system information (like /proc/) or something like /dev/null/, so I used std::filesystem::directory_options::skip_permission_denied in my for loop to skip permission issues and I used multiple try/catch blocks to catch exceptions.
However, even with this, a permission denied exception is still thrown. See the following code:
byte_size_and_num_files find_recursive(const std::filesystem::path& path)
{
byte_size_and_num_files bsnf;
std::filesystem::path pa;
try
{
for(const auto& p: std::filesystem::recursive_directory_iterator(path, std::filesystem::directory_options::skip_permission_denied))
{
pa = p.path();
if (std::filesystem::exists(p) && !std::filesystem::is_directory(p))
{
try
{
if(std::filesystem::is_regular_file(pa))
{
bsnf.size += std::filesystem::file_size(p);
bsnf.files++;
}
else
std::cout << "SKIPPED: size is not determinable: " << pa << "\n";
}
catch(std::filesystem::filesystem_error& e)
{
std::cout << "SKIPPED: size is not determinable: " << pa << "\n";
}
catch(std::bad_alloc)
{
std::cout << "Allocation error. Exiting..." << "\n";
byte_size_and_num_files err;
return err;
}
}
}
}
catch(std::filesystem::filesystem_error& e)
{
std::cout << "Unable to access file or path " << pa <<": " << e.what() << "\n";
}
catch(std::bad_alloc)
{
std::cout << "Allocation error. Exiting..." << "\n";
byte_size_and_num_files err;
return err;
}
return bsnf;
}
Output:
...
SKIPPED: size is not determinable: "/lib/systemd/system/motd.service"
SKIPPED: size is not determinable: "/lib/systemd/system/mountall-bootclean.service"
SKIPPED: size is not determinable: "/lib/systemd/system/mountall.service"
SKIPPED: size is not determinable: "/lib/systemd/system/mountdevsubfs.service"
SKIPPED: size is not determinable: "/lib/systemd/system/mountkernfs.service"
SKIPPED: size is not determinable: "/lib/systemd/system/mountnfs-bootclean.service"
SKIPPED: size is not determinable: "/lib/systemd/system/mountnfs.service"
SKIPPED: size is not determinable: "/lib/systemd/system/rc.service"
SKIPPED: size is not determinable: "/lib/systemd/system/rcS.service"
SKIPPED: size is not determinable: "/lib/systemd/system/reboot.service"
SKIPPED: size is not determinable: "/lib/systemd/system/rmnologin.service"
SKIPPED: size is not determinable: "/lib/systemd/system/sendsigs.service"
SKIPPED: size is not determinable: "/lib/systemd/system/single.service"
SKIPPED: size is not determinable: "/lib/systemd/system/stop-bootlogd-single.service"
SKIPPED: size is not determinable: "/lib/systemd/system/stop-bootlogd.service"
SKIPPED: size is not determinable: "/lib/systemd/system/umountfs.service"
SKIPPED: size is not determinable: "/lib/systemd/system/umountnfs.service"
SKIPPED: size is not determinable: "/lib/systemd/system/umountroot.service"
SKIPPED: size is not determinable: "/lib/systemd/system/x11-common.service"
SKIPPED: size is not determinable: "/lib/systemd/system/lvm2.service"
Unable to access file or path "/proc/1/task/1/cwd": filesystem error: status: Permission denied [/proc/1/task/1/cwd]
/ is 262172.00 gigabytes with 143839 files.
What gives? Its clear permission denied should always be skipped, but it is still thrown. It is also clear that files whose size cannot be determined are also skipped, but it appears as though some incorrect file sizes are found and used (262172.00 GB is certainly not correct). This ONLY occurs when going through the root directory, never any other directory like home.
Could this be an implementation error?
EDIT: Some debugging revealed some important information. It appears as though /dev/pts/ specifically ruins file sizes. I do not understand why the files in this directory are not skipped, as they are not regular files so they should be skipped. I suppose std::filesystem incorrectly identifies these as regular files.
In addition, std::filesystem::directory_options::skip_permission_denied DOES work for every single file except /proc/1/task/1/cwd. I am not sure why this file specifically is not skipped.
EDIT 2 I have tracked down the culprit directories that break std::filesystem. There are three,
/dev/
/proc/
/var/cache/
/dev/ will mess up file sizes for whatever reason, I guess std::filesystem considers these regular files so when you try to fetch file sizes you get a garbage number. Some parts of /dev/ will throw a permission exception EVEN WHEN skip permission is used.
/proc/ also messes up file sizes for the same reason as /dev/
/var/cache/ does not appear to mess up file sizes but will throw permission exception even when skip permission is in use.
I believe these to be implementation errors, as the files in these directories are not regular files but std::filesystem treats them as such.
The docs on cppreference say that skip_permission_denied is to "Skip directories that would otherwise result in permission denied errors.". It says nothing about files, only about directories. The option means that in case you cannot read "/proc/sys/xyz" this directory is skipped and no exception is thrown. In your case this is respected and you do not get a filesystem_error exception from recursive_directory_iterator. But that is not related to any file that this iterator returns.
But an exception is thrown in std::filesystem::exists(p) and in std::filesystem::is_directory(p). It is thrown by std::filesystem::status which is internally called by both functions. en.cppreference says that in status "symlinks are followed to their targets". This is not possible for /proc/1/task/1/cwd if you are not root.
ls -alg /proc/1/task/1/cwd shows me a file mode lrwxrwxrwx but a permission denied message.
$ ls -alg /proc/1/task/1/cwd
ls: Lesen der symbolischen Verknüpfung '/proc/1/task/1/cwd' nicht möglich: Keine Berechtigung
lrwxrwxrwx 1 root 0 Mär 11 08:53 /proc/1/task/1/cwd
I believe that the recursive_directory_iterator is only looking at the file mode, but cwd seems to be special and additional access rules apply. So a solution for you could be to catch exceptions thrown by std::filesystem::exists or std::filesystem::is_directory, ignore them and proceed.