Search code examples
pythonrlinuxcondarpy2

How to select R installation when using rpy2 on conda?

tl;dr Can I configure rpy2 so that it loads the conda installation instead of the system one?

My issue

When I run rpy2 it uses the system installation of R, but it introspects and loads libraries from my conda installation of R. This has various side effects such as segfaults and so on...

Reproduction

I have an installation of R on a Ubuntu 18 server, that my sysadmin has installed. I want to use conda with python and rpy2 to make sure I can select versions somewhat freely.

Below, you can see I am in a conda environment called (rnvp), that rpy2 identifies the 4.0.5 conda installation of R, but that the rpy2.robjects library loads the system R installation instead, which is 4.0.3.

When running the situation script, I get the below

(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ python
Python 3.9.6 (default, Aug 18 2021, 19:38:01) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpy2.robjects as ro
During startup - Warning messages:
1: package ‘methods’ was built under R version 4.0.5 
2: package ‘datasets’ was built under R version 4.0.5 
3: package ‘utils’ was built under R version 4.0.5 
4: package ‘grDevices’ was built under R version 4.0.5 
5: package ‘graphics’ was built under R version 4.0.5 
6: package ‘stats’ was built under R version 4.0.5 
>>> print(ro.r)
<rpy2.robjects.R object at 0x7f8f16ece8b0>
platform: x86_64-pc-linux-gnu
arch: x86_64
os: linux-gnu
system: x86_64, linux-gnu
status: 
major: 4
minor: 0.3
year: 2020
month: 10
day: 10
svn rev: 79318
language: R
version.string: R version 4.0.3 (2020-10-10)
nickname: Bunny-Wunnies Freak Out
>>> import rpy2.situation
>>> for row in rpy2.situation.iter_info():
...     print(row)
... 
rpy2 version:
3.4.5
Python version:
3.9.6 (default, Aug 18 2021, 19:38:01) 
[GCC 7.5.0]
Looking for R's HOME:
    Environment variable R_HOME: /home/ludhu126/miniconda3/envs/rnvp/lib/R
    Calling `R RHOME`: /home/ludhu126/miniconda3/envs/rnvp/lib/R
    Environment variable R_LIBS_USER: None
R's additions to LD_LIBRARY_PATH:

R version:
    In the PATH: R version 4.0.5 (2021-03-31) -- "Shake and Throw"
    Loading R library from rpy2: OK
Additional directories to load R packages from:
None
C extension compilation:
  include:
  ['/home/ludhu126/miniconda3/envs/rnvp/lib/R/include']
  libraries:
  ['R', 'pcre2-8', 'lzma', 'bz2', 'z', 'rt', 'dl', 'm', 'iconv', 'icuuc', 'icui18n']
  library_dirs:
  ['/home/ludhu126/miniconda3/envs/rnvp/lib', '/home/ludhu126/miniconda3/envs/rnvp/lib/R/lib', '/home/ludhu126/miniconda3/envs/rnvp/lib']
  extra_compile_args:
  []
  extra_link_args:
  ['-Wl,--export-dynamic', '-fopenmp', '-Wl,-O2', '-Wl,--sort-common', '-Wl,--as-needed', '-Wl,-z,relro', '-Wl,-z,now', '-Wl,--disable-new-dtags', '-Wl,--gc-sections', '-Wl,-rpath,/home/ludhu126/miniconda3/envs/rnvp/lib', '-Wl,-rpath-link,/home/ludhu126/miniconda3/envs/rnvp/lib']

The terminal points to the conda installation of R, which indeed is 4.0.5

(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ which R
/home/ludhu126/miniconda3/envs/rnvp/bin/R
(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ R --version
R version 4.0.5 (2021-03-31) -- "Shake and Throw"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

Outside this environment, I have R 4.0.3 from /usr/lib/R

(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ conda activate base
(base) ludhu126@hyperion:~/real-nvp/real-nvp$ which R
/usr/bin/R
(base) ludhu126@hyperion:~/real-nvp/real-nvp$ R --version
R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

Question Can I configure rpy2 so that it loads the conda installation instead?

Close duplicates

Solution

  • Recompile rpy2 locally, with a linker flag to explicate which R library to link to.

    Inspired by the blog post https://thomas-cokelaer.info/blog/2012/01/installing-rpy2-with-different-r-version-already-installed/ I decided to recompile rpy2 from source, and supply the R path as an environment variable to the compilation.

    (rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ export LDFLAGS="-Wl,-rpath,/home/ludhu126/miniconda3/envs/rnvp/lib/R/lib"
(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ pip install rpy2 --force-reinstall --compile --no-binary rpy2
Collecting rpy2
  Using cached rpy2-3.4.5.tar.gz (194 kB)
Collecting cffi>=1.10.0
  Using cached cffi-1.14.6-cp39-cp39-manylinux1_x86_64.whl (405 kB)
Collecting jinja2
  Using cached Jinja2-3.0.1-py3-none-any.whl (133 kB)
Collecting pytz
  Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)
Collecting tzlocal
  Using cached tzlocal-3.0-py3-none-any.whl (16 kB)
Collecting pycparser
  Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting MarkupSafe>=2.0
  Using cached MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB)
Skipping wheel build for rpy2, due to binaries being disabled for it.
Installing collected packages: pycparser, MarkupSafe, tzlocal, pytz, jinja2, cffi, rpy2
  Attempting uninstall: pycparser
    Found existing installation: pycparser 2.20
    Uninstalling pycparser-2.20:
      Successfully uninstalled pycparser-2.20
  Attempting uninstall: MarkupSafe
    Found existing installation: MarkupSafe 2.0.1
    Uninstalling MarkupSafe-2.0.1:
      Successfully uninstalled MarkupSafe-2.0.1
  Attempting uninstall: tzlocal
    Found existing installation: tzlocal 3.0
    Uninstalling tzlocal-3.0:
      Successfully uninstalled tzlocal-3.0
  Attempting uninstall: pytz
    Found existing installation: pytz 2021.1
    Uninstalling pytz-2021.1:
      Successfully uninstalled pytz-2021.1
  Attempting uninstall: jinja2
    Found existing installation: Jinja2 3.0.1
    Uninstalling Jinja2-3.0.1:
      Successfully uninstalled Jinja2-3.0.1
  Attempting uninstall: cffi
    Found existing installation: cffi 1.14.6
    Uninstalling cffi-1.14.6:
      Successfully uninstalled cffi-1.14.6
  Attempting uninstall: rpy2
    Found existing installation: rpy2 3.4.5
    Uninstalling rpy2-3.4.5:
      Successfully uninstalled rpy2-3.4.5
    Running setup.py install for rpy2 ... done
Successfully installed MarkupSafe-2.0.1 cffi-1.14.6 jinja2-3.0.1 pycparser-2.20 pytz-2021.1 rpy2-3.4.5 tzlocal-3.0
(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ python
Python 3.9.6 (default, Aug 18 2021, 19:38:01) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpy2.robjects as ro; print(ro.r('version'))
               _                           
platform       x86_64-conda-linux-gnu      
arch           x86_64                      
os             linux-gnu                   
system         x86_64, linux-gnu           
status                                     
major          4                           
minor          0.5                         
year           2021                        
month          03                          
day            31                          
svn rev        80133                       
language       R                           
version.string R version 4.0.5 (2021-03-31)
nickname       Shake and Throw             
>>>