Search code examples
rpackagepackratrenv

How to Manage R Packages given Windows 255 file path limit, e.g. checkpoint and Rcpp?

So I was trying to install Rcpp using the checkpoint package (with a March 1st 2020 date).

Most of my packages were fine, but Rcpp specifically makes a lot of temporary directories that it then deletes, for example: 

00LOCK-Rcpp/00new/Rcpp/include/Rcpp/generated/InternalFunctionWithStdFunction_call.h

This is 84 characters long and I belive some are longer.

Checkpoint creates numerous directories as well, for example with a custom library here: 

"custom_library/.checkpoint/2020-03-01/lib/x86_64-w64-mingw32/3.6.0/"

This is 67 characters, of which 52 are only necessary when managing multiple checkpoint dates or versions.

This means that for a file path such as: 

"C:/Users/USER/OneDrive - COMPANY/Documents/LargeDirectory/SubDirectory1/SubDirectory2/custom_library/.checkpoint/2020-03-01/lib/x86_64-w64-mingw32/3.6.0/Rcpp"

Assuming even temporary files can't exceed 255 chars then I have definitely < 60 characters left available for all of the Rcpp temporary objects.

I tested with the following code: 

setwd("C:/Users/USERNAME/OneDrive - COMPANY/Documents/LargeDirectory/SubDirectory1/SubDirectory2/") 
dir.create("custom_library)
checkpoint(as.Date("2020-03-01"),
           checkpointLocation = paste0(
          "SubDirectory2","/custom_library")
          )
y
install.packages("Rcpp")

It fails because of numerous "no file or directory found" which I believe actually fails because 00LOCK-Rcpp/00new/Rcpp/include/Rcpp/ can't be created to then unzip all the .h files to it. I was curious so I ran the following: 

setwd("~") # up to Documents 
dir.create("Rcpptest")
.libPaths("Rcpptest")
install.packages("Rcpp")

And it installed fine.

Any ideas on how to make checkpoint either not create so many nested directories OR ignore the file_path 255 limit until the whole package installs?

For now, I will likely move the directory up a few levels but any insight into whether my guess is actually correct or if I'm missing something would be appreciated!

Solution

  • I believe you are correct -- this is, to the best of my knowledge, a limitation of the internal unzip implementation used by R, which is ultimately a limitation of the Windows APIs used by R. See https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file for some more discussion.

    There are a couple options for mitigating the issue that may be worth trying.

    1. Use utils::shortPathName() to construct a so-called Windows 'short path'. This will help trim longer path components and bring the full path down in size.

    2. Create a junction to your project using Sys.junction() to a local path with a shorter length, and move to that directory. See ?Sys.junction for more information -- a junction is basically like a Windows shortcut, or a symlink to a directory.

    In each case, you should hopefully be able to construct a path that is "identical" to your current project directory, but short enough that things can function as expected.