While testing an R package install on Windows I found that package dependency resolution that used to work now fails with package version mismatch errors. It looks like this happens for cases where one package is available as source without compilation needed, while a dependency's source package would need compilation, so it falls back on the (older) binary package for the dependency. But in doing that it keeps the newer version of the other package, so its dependencies aren't met, and the install fails.
To give one example, with tinytex as the package and xfun as the dependency:
> utils::install.packages("tinytex", repos = "https://cloud.r-project.org")
Installing package into ‘C:/Users/User/Documents/R/win-library/3.6’
(as ‘lib’ is unspecified)
also installing the dependency ‘xfun’
There are binary versions available but the source versions are later:
binary source needs_compilation
xfun 0.22 0.32 TRUE
tinytex 0.31 0.41 FALSE
Binaries will be installed
trying URL 'https://cloud.r-project.org/bin/windows/contrib/3.6/xfun_0.22.zip'
Content type 'application/zip' length 323923 bytes (316 KB)
downloaded 316 KB
package ‘xfun’ successfully unpacked and MD5 sums checked
The downloaded binary packages are in
C:\Users\User\AppData\Local\Temp\Rtmp6xfEaC\downloaded_packages
installing the source package ‘tinytex’
trying URL 'https://cloud.r-project.org/src/contrib/tinytex_0.41.tar.gz'
Content type 'application/x-gzip' length 32910 bytes (32 KB)
downloaded 32 KB
* installing *source* package 'tinytex' ...
** package 'tinytex' successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]) :
namespace 'xfun' 0.22 is being loaded, but >= 0.29 is required
Calls: <Anonymous> ... namespaceImportFrom -> asNamespace -> loadNamespace
Execution halted
ERROR: lazy loading failed for package 'tinytex'
* removing 'C:/Users/User/Documents/R/win-library/3.6/tinytex'
The downloaded source packages are in
‘C:\Users\User\AppData\Local\Temp\Rtmp6xfEaC\downloaded_packages’
Warning message:
In utils::install.packages("tinytex", repos = "https://cloud.r-project.org") :
installation of package ‘tinytex’ had non-zero exit status
What I don't understand is why it would install an incompatible combination of packages, rather than installing the latest available (compatible) combination. In this example, tinytex 0.31 only requires xfun at or above 0.19, so both older binary packages would work, and I can force this behavior with type = "binary" during the install. But shouldn't install.packages recognize that the newest tinytex's dependency isn't available, rather than break during install? Is this the expected behavior on Windows, or a bug?
(This is with R 3.6.3 on Windows 11.)
To clarify about the older R: I realize this is the previous rather than latest major release, but it's the dependency-handling behavior I'm trying to figure out here, since this could crop up with any R version depending on what combinations of binary and source packages happen to be available. How should I ensure this doesn't happen? And should this be reported as a bug in the dependency-handling logic since (as far as I can tell) the combination of packages selected doesn't make sense?
I reported this as bug #18396 on 2022-08-30. We'll see what the verdict is.
I think this is a bug in the package installer, and it persists in the latest versions of R, including devel. That it came up here with old R is just a consequence of older package versions revealing the problem, but it looks like the utils package itself is to blame. I'll try to point this out to the maintainers.
In more detail:
In the source for the utils package, install.packages calls getDependencies (both in packages2.R)which in turn calls .clean_up_dependencies2 (in packages.R), which is what actually checks version requirements against available packages. That private function divides the dependent package names into those that can be satisfied and those that can't. If the available package versions satisfy all dependencies, that returns a vector of those names to install, plus an (empty) vector of the missing dependencies.
But the handling of what version is needed to satisfy what dependency ends there; it's just package names after that. Then, later in install.packages, there's the additional logic to select different versions based on which are available as source/binary... but if that selection is altered, it can break what was previously a consistent set of package versions as discovered by .clean_up_dependencies2. (That function even has a comment near the end that says install.packages() will find the highest version, but that might not be the case depending on what happens later on.)
That's the bug: the earlier logic thinks the dependencies are satisfied by the available package versions, but the later logic alters that set of versions. I think the expected behavior would be that the package type-handling would happen first, followed by the version-aware dependency-handling.
To clarify what I think is the bug and why:
If install.packages("packageA", type="both", ...) runs into this situation:
There are binary versions available but the source versions are later:
binary source needs_compilation
packageB 1.0 2.0 TRUE
packageA 1.0 2.0 FALSE
...where packageA 2.0 depends on packageB 2.0, what is the expected behavior? Should it try to install 1.0 of packageB and 2.0 of packageA and fail, as it currently does? Or should it install 1.0 of both since that's the latest compatible combination of available packages? (If it's the latter, this answer is right and this is a bug. If it's the former, I'm wrong and this failure is actually the expected behavior.)