Search code examples
pythonpipcryptographypycryptopyopenssl

ImportError: ./cryptography/src/cryptography/hazmat/bindings/_rust.abi3.so: undefined symbol: EVP_PKEY_get_id when using Cryptography

I am developing a Python project that uses Cryptography and PyOpenSSL. I created a virtual environment for all the project's dependencies based on Python 3.10. Since I need some more OpenSSL functionality than what these libraries provide, I modified the libraries, then built them with pip install -e . (in the venv). I then ran my project and it worked perfectly, including the new functionality I added to the libraries. Now, I want to run the same project on a different device, so I pushed my project, as well as the Cryptography library and the PyOpenSSL library to three separate GitLab Repo and cloned them from my laptop. I included the venv folder in the Git Project, because I assumed that would make it easier to run on my second device.

I cloned the three projects and ran my project, however, I encountered the following error:

ImportError: /home/xy/Projects/cryptography/src/cryptography/hazmat/bindings/_rust.abi3.so: undefined symbol: EVP_PKEY_get_id

I tried installing the two libraries again (from the cloned Git Projects), but the error persists. I checked that OpenSSL is correctly installed with openssl version and it seems like it is. I assume there is some problem with the venv, so I tried the same without a venv: Installed the libraries, then ran the project, but the problem persists.

I assume this is not some problem with my Code, since it is running fine on my Desktop, but rather a problem with pip installing the libraries. Both devices run KDE Neon, Python 3.10.12 and OpenSSL 3.0.2. Does someone have a fix for this? Does it make sense to share the venv via Git or should I create a new venv? Did I make any mistake when installing?

Edit: I tried using virtualenv instead of venv like suggested here, but it still does not work. Also, I figured that EVP_PKEY_get_id was added in OpenSSL 3.0. So I assume the problem is that I compile cryptography against OpenSSL 3.0 but then its linked against OpenSSL 1.x (as described here). I am not sure how to fix this though. I noticed that in /usr/lib/x86-64-linux-gnu there are the following files

libcrypt.so libcrypt.so.1 libcrypt.so.1.1.0

and

libssl.so libssl.so.1.1 libssl.so.3

I wonder whether the 1.1 versions of these are used somehow and cause the problem? I am not sure how to check that though and I'm hesitant to just delete those.

Solution

  • I found a solution that worked for me: The Docs provide a script to build a static wheel. In my venv, I ran pip uninstall cryptography, then created an empty .sh file, pasted the script below in there and ran it. This created a .whl file, that I then installed with pip install <.whl_file>

    Script (slightly modified for my local cryptography version and OpenSSL 3.0.2:

    set -e

OPENSSL_VERSION="3.0.2"
CWD=$(pwd)

virtualenv env
. env/bin/activate
pip install -U setuptools
pip install -U wheel pip
curl -O https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
tar xvf openssl-${OPENSSL_VERSION}.tar.gz
cd openssl-${OPENSSL_VERSION}
./config no-shared no-ssl2 no-ssl3 -fPIC --prefix=${CWD}/openssl
make && make install
cd ..
OPENSSL_DIR="${CWD}/openssl" pip wheel --no-cache-dir --no-binary cryptography ./