I'm trying to build an Docker image which should run a python script, which needs numpy, scipy, pandas and google-cloud-bigquery.
Since this image is build for an armv7 architecture it's a pain to install numpy, scipy and pandas directly (it takes too long and finally it breaks). So I decided to use Miniconda and used the packeges for Raspberry Pi. That worked fine (installation can be completet during image build).
Now I'm trying to install the google packages google-crc32c==1.1.2 and google-cloud-bigquery. With pip this is possible and the image is build properly. But if I run a container with this Image it is always restarting and gives me this error log:
File "/usr/src/app/bigquery.py", line 1, in <module>
from google.cloud import bigquery
ImportError: No module named 'google'
I think I have to install the google packages with conda but there are no packages for armv7 architecture available:
google-cloud-bigquery package on Anaconda.org: https://anaconda.org/search?q=google+bigquery
google-crc32c package on Anaconda.org: https://anaconda.org/search?q=google-crc32c
Is there a possibility to install those google packages with Miniconda for armv7 architecture? Or is another way possible to install numpy, scipy and pandas without using miniconda (but not installing them directly)?
Thank you for any help!
Dockerfile:
FROM python:3.7-buster
WORKDIR /usr/src/app
ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"
COPY main_prog.py bigquery.py requirements.txt ./
RUN wget http://repo.continuum.io/miniconda/Miniconda3-latest-Linux-armv7l.sh
RUN mkdir /root/.conda
RUN /bin/bash Miniconda3-latest-Linux-armv7l.sh -b
RUN rm -f Miniconda3-latest-Linux-armv7l.sh \
&& echo "Running $(conda --version)"
RUN wget https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh
RUN chmod +x Berryconda3-2.0.0-Linux-armv7l.sh ./Berryconda3-2.0.0-Linux-armv7l.sh
RUN conda list \
&& conda config --add channels rpi \
&& conda install python=3.6 -y\
&& conda install openblas blas -y\
&& conda install numpy -y\
&& conda install pandas -y\
&& conda install scipy -y
RUN pip install --upgrade pip
RUN pip install "google-crc32c==1.1.2"
RUN pip install google-cloud-bigquery
CMD ["python", "main_prog.py"]
I couldn't find I way to install all packages with Miniconda.
But it was possible for me to install them directly with wheels from piwheels.
Therefor I had to add a pip.conf file in "/etc" dirctory.
content of pip.conf:
[global]
extra-index-url=https://www.piwheels.org/simple
In addition I had to install libatlas-base-dev.
I only could do this by adding an URL deb http://ftp.de.debian.org/debian buster main (like it's recommended here) to my sources.list in "/etc/apt/" directory.
content of sources.list:
# deb http://snapshot.debian.org/archive/debian/20210902T000000Z buster main
deb http://deb.debian.org/debian buster main
# deb http://snapshot.debian.org/archive/debian-security/20210902T000000Z buster/updates main
deb http://security.debian.org/debian-security buster/updates main
# deb http://snapshot.debian.org/archive/debian/20210902T000000Z buster-updates main
deb http://deb.debian.org/debian buster-updates main
deb http://ftp.de.debian.org/debian buster main
Dockerfile:
FROM python:3.7-buster
WORKDIR /usr/src/app
COPY main_prog.py bigquery.py requirements.txt pip.conf sources.list ./
RUN mv ./pip.conf /etc \
&& export PIP_CONFIG_FILE=/etc/pip.conf
RUN mv ./sources.list /etc/apt/
RUN apt-get update \
&& apt-get upgrade -y
RUN apt-get install libatlas-base-dev -y
RUN pip3 install --upgrade pip
RUN pip3 install numpy \
&& pip3 install scipy \
&& pip3 install pandas \
&& pip3 install google-crc32c \
&& pip3 install google-cloud-bigquery
CMD ["python", "main_prog.py"]