Search code examples
pythondockerrustalpine-linuxorjson

How do I setup rust toolchain for orjson python library build in an alpine docker image?


I have been struggling with this and I need some help. I am trying to create a docker image of my python app. My App uses orjson, the fast JSON library for python. Parts of this library is built using rust so I need the rust toolchain. Now the kicker is that I am using Alpine for its small footprint so it comes with none of the standard tools included. I need set things up myself.

Here is my Dockerfile to simulate the problem.

FROM python:3.7-alpine

# for orjson in requirements.txt
RUN apk add rust cargo


RUN pip install orjson

This is the error message that I am facing.

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM python:3.7-alpine
 ---> 6a5ca85ed89b
Step 2/3 : RUN apk add rust cargo
 ---> Running in b86315a52e50
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
(1/19) Installing rust-stdlib (1.43.1-r1)
(2/19) Installing libgcc (9.3.0-r2)
(3/19) Installing libstdc++ (9.3.0-r2)
(4/19) Installing binutils (2.34-r1)
(5/19) Installing gmp (6.2.0-r0)
(6/19) Installing isl (0.18-r0)
(7/19) Installing libgomp (9.3.0-r2)
(8/19) Installing libatomic (9.3.0-r2)
(9/19) Installing libgphobos (9.3.0-r2)
(10/19) Installing mpfr4 (4.0.2-r4)
(11/19) Installing mpc1 (1.1.0-r1)
(12/19) Installing gcc (9.3.0-r2)
(13/19) Installing musl-dev (1.1.24-r8)
(14/19) Installing libxml2 (2.9.10-r4)
(15/19) Installing llvm10-libs (10.0.0-r2)
(16/19) Installing rust (1.43.1-r1)
(17/19) Installing nghttp2-libs (1.41.0-r0)
(18/19) Installing libcurl (7.69.1-r0)
(19/19) Installing cargo (1.43.1-r1)
Executing busybox-1.31.1-r16.trigger
OK: 334 MiB in 54 packages
Removing intermediate container b86315a52e50
 ---> 07671da6f533
Step 3/3 : RUN pip install orjson
 ---> Running in 9c72ff2b2e3e
Collecting orjson
  Downloading orjson-3.0.2.tar.gz (649 kB)
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpxhkdxsol
         cwd: /tmp/pip-install-0x1gorur/orjson
    Complete output (14 lines):
    💥 maturin failed
      Caused by: Cargo metadata failed. Do you have cargo in your PATH?
      Caused by: Error during execution of `cargo metadata`: error: failed to run `rustc` to learn about target-specific information

    Caused by:
      process didn't exit successfully: `rustc - --crate-name ___ --print=file-names -Z mutable-noalias --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit code: 1)
    --- stderr
    error: the option `Z` is only accepted on the nightly compiler



    Checking for Rust toolchain....
    Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-7txy00_d --manylinux=off --strip=on`
    Error: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-7txy00_d', '--manylinux=off', '--strip=on']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpxhkdxsol Check the logs for full command output.
The command '/bin/sh -c pip install orjson' returned a non-zero code: 1

I am a bit lost on what I should install for getting a rust environment in Alpine. Any help would be appreciated.


Solution

  • I managed to get it to work thanks to @nitishkumar-singh response. There were some errors in that post here is my Dockerfile if someone runs into this issue.

    FROM python:3.7-alpine3.11
    
    RUN echo "https://dl-3.alpinelinux.org/alpine/v3.11/main" >> /etc/apk/repositories
    RUN echo "https://dl-3.alpinelinux.org/alpine/v3.11/community" >> /etc/apk/repositories
    RUN apk add --no-cache python3 gcompat patchelf
    RUN patchelf --add-needed libgcompat.so.0 /usr/bin/python3
    RUN echo 'manylinux1_compatible = True' > /usr/local/lib/python3.7/_manylinux.py &&\
    pip3 install orjson &&\
    rm /usr/local/lib/python3.7/_manylinux.py