help stepI've encountered an issue when cross-compiling the R package xml2 on the Conda Forge infrastructure (r-xml2-feedstock). The Conda Forge *nix builds for R execute with:
R CMD INSTALL --no-test-load --build .
where the --no-test-load flag (normally) helps avoid any issues with cross-compilation. The building of the dynamic library, including linking to a target-native libxml2, works fine. However, the package installation fails during the help step:
* installing *source* package ‘xml2’ ...
** package ‘xml2’ successfully unpacked and MD5 sums checked
** using staged installation
Found INCLUDE_DIR and/or LIB_DIR!
Using PKG_CFLAGS=-I$PREFIX/include/libxml2
Using PKG_LIBS=-L$PREFIX/lib -lxml2
** libs
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c connection.cpp -o connection.o
arm64-apple-darwin20.0.0-clang -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c init.c -o init.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_doc.cpp -o xml2_doc.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_init.cpp -o xml2_init.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_namespace.cpp -o xml2_namespace.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_node.cpp -o xml2_node.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_output.cpp -o xml2_output.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_schema.cpp -o xml2_schema.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_url.cpp -o xml2_url.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -I"$BUILD_PREFIX/lib/R/include" -DNDEBUG -I../inst/include -I$PREFIX/include/libxml2 -DUCHAR_TYPE=wchar_t -DU_SHOW_CPLUSPLUS_API=0 -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0 -I$PREFIX/include -fPIC -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base-split_1648746613541/work=/usr/local/src/conda/r-base-4.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -c xml2_xpath.cpp -o xml2_xpath.o
arm64-apple-darwin20.0.0-clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L$PREFIX/lib/R/lib -Wl,-dead_strip_dylibs -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib -o xml2.dylib connection.o init.o xml2_doc.o xml2_init.o xml2_namespace.o xml2_node.o xml2_output.o xml2_schema.o xml2_url.o xml2_xpath.o -L$PREFIX/lib -lxml2 -L$PREFIX/lib/R/lib -lR -Wl,-framework -Wl,CoreFoundation
ld: warning: -pie being ignored. It is only used when linking a main executable
installing to /Users/runner/miniforge3/conda-bld/r-xml2_1649126088207/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/R/library/00LOCK-work/00new/xml2/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
Error : unable to load shared object '/Users/runner/miniforge3/conda-bld/r-xml2_1649126088207/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/R/library/00LOCK-work/00new/xml2/libs/xml2.dylib':
dlopen(/Users/runner/miniforge3/conda-bld/r-xml2_1649126088207/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/R/library/00LOCK-work/00new/xml2/libs/xml2.dylib, 6): no suitable image found. Did find:
/Users/runner/miniforge3/conda-bld/r-xml2_1649126088207/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/R/library/00LOCK-work/00new/xml2/libs/
ERROR: installing Rd objects failed for package ‘xml2’
Obviously, this fails because the .dylib is not compiled for the host architecture, so we wouldn't expect it to be loadable. The question is: Why is the help installation step attempting to load the shared library? and how can this be avoided?
I've confirmed that it really is just the help step that triggers this self-loading. The package build works just fine with addition of a --no-help flag. However, this is not ideal - it would be preferable to deliver help manuals to all platforms.
Attempting to exclude only specific parts of the help did not avoid the self-loading error, namely, each of the --no-docs and --no-html flags alone still failed.
I suspect something in the .Rd files (examples?) is triggering R to attempt to load the library. However, I've not encountered this issue with other R packages that use Roxygen2-generated .Rd files as help documentation. For example, nloptr (package, feedstock) cross-compiles successfully. Any insights into what to look for would be appreciated.
Some packages need to import itself when building the docs. In the case of xml2, the xml options are loaded from the R package and then injected to the docs. See https://github.com/r-lib/xml2/blob/ab2078976c9ab57116c7c9175632e7d19d45273f/man/read_xml.Rd#L58
To fix this, you'll have to compile the package twice. One for cross architecture (Apple Silicon) and one for native architecture (Intel) and use the native package to build the docs assuming that the docs generated are the same for both architecture. There might an example or two in conda-forge that does this.