I am creating a Dockerfile to run R workloads in Lambda based on a template from the lambdr package, which relies on an Amazon Linux base image and yum package manager. I need glmnet for one of the workloads I want to run. However, I can't get to install glmnet as it seems like there is some issue with the compilation.
This is the beginning of my Dockerfile:
FROM public.ecr.aws/lambda/provided:al2-x86_64
ENV R_VERSION=4.4.0
# Installs wget, git, tar, gcc and other development libraries
RUN yum -y install wget git tar gcc gcc-c++ libcurl-devel openssl-devel libxml2-devel
# Install EPEL
RUN yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \
# Install specific R version
&& wget https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm \
&& yum -y install R-${R_VERSION}-1-1.x86_64.rpm \
&& rm R-${R_VERSION}-1-1.x86_64.rpm
# Install AWS CLI
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
&& unzip awscliv2.zip \
&& ./aws/install
# Accept AWS credentials as build arguments
ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
# Set AWS credentials in the environment
ENV AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
ENV AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
ENV PATH="${PATH}:/opt/R/${R_VERSION}/bin/"
# Creates a configuration file for R to set the C++ compiler.
RUN mkdir -p /root/.R && echo "CXX17 = g++" >> /root/.R/Makevars
RUN Rscript -e "install.packages('glmnet', \
INSTALL_opts =c('--no-docs', '--no-data',\
'--no-help', '--no-demo'),\
repos = 'https://packagemanager.rstudio.com/all/__linux__/centos7/latest')"
I then call
docker build --build-arg AWS_ACCESS_KEY_ID={AWS_ACCESS_KEY_ID}--build-arg AWS_SECRET_ACCESS_KEY={AWS_SECRET_ACCESS_KEY} -t r-on-lambda .
Which outputs this at the end:
#11 107.7 /usr/bin/ld: RcppExports.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: elnet_exp.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: glmnet_init.o: relocation R_X86_64_32 against `.data' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: internal.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: pb_exp.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: wls_exp.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: final link failed: Nonrepresentable section on output
#11 107.7 collect2: error: ld returned 1 exit status
#11 107.7 make: *** [glmnet.so] Error 1
#11 107.7 ERROR: compilation failed for package ‘glmnet’
#11 107.7 * removing ‘/opt/R/4.4.0/lib/R/library/glmnet’
#11 107.8
#11 107.8 The downloaded source packages are in
#11 107.8 ‘/tmp/Rtmp7LKwru/downloaded_packages’
#11 107.8 Updating HTML index of packages in '.Library'
#11 107.8 Making 'packages.html' ... done
#11 107.8 Warning message:
#11 107.8 In install.packages("glmnet", INSTALL_opts = c("--no-docs", "--no-data", :
#11 107.8 installation of package ‘glmnet’ had non-zero exit status
#11 DONE 107.9s
What is the reason for glmnet not being installed if I have gcc and gcc-c++ installed? In the log I have seen that all the dependencies (foreach, iterators, Rcpp, RcppEigen, shape) do get installed properly. How can I solve this?
I've discovered you can add the -fPIC flag into the CXX17 required by glmnet:
Modifying this:
RUN mkdir -p /root/.R && echo "CXX17 = g++" >> /root/.R/Makevars
For that:
RUN mkdir -p /root/.R && \
echo "CXX17 = g++ -fPIC" >> /root/.R/Makevars && \
echo "CXX17FLAGS = -O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function" >> /root/.R/Makevars