Issue using M2Crypto on lambda (works on EC2)

I am trying to install a python function using M2Crypto in AWS Lambda.

I spun up an EC2 instance with the Lambda AMI image, installed M2Crypto into a virtualenv, and was able to get my function working on EC2.

Then I zipped up the site-package and uploaded to Lambda. I got this error

Unable to import module 'epd_M2Crypto': /var/task/M2Crypto/_m2crypto.cpython-36m-x86_64-linux-gnu.so: symbol sk_deep_copy, version libcrypto.so.10 not defined in file libcrypto.so.10 with link time reference

There are similar questions and hints here and here. I tried uploading the offending lib (libcrypto.so.10) in the zip file, but still get the same error. I am assuming the error means that the EC2 version of libcrypto.so.10 (used to install M2Crypto) is different than the version on Lambda (that I trying to run with), so M2Crypto complains.

If I look at the versions of openssl they are different:

OpenSSL 1.0.0-fips 29 Mar 2010 (lambda version)
OpenSSL 1.0.2k-fips 26 Jan 2017 (ec2 version)

I don't think the answer is to downgrade openssl on ec2 as the 1.0.0 version is obsolete (AWS applies security patches but the version still shows as 1.0.0). (Also the yum doesn't have versions this old)

Here's the steps i used on the EC2 instance to get it working on EC2:

$ sudo yum -y update
$ sudo yum -y install python36
$ sudo yum -y install python-virtualenv
$ sudo yum -y groupinstall "Development Tools"
$ sudo yum -y install python36-devel.x86_64
$ sudo yum -y install openssl-devel.x86_64

$ mkdir ~/forlambda
$ cd ~/forlambda
$ virtualenv -p python3 venv
$ source venv/bin/activate

$ cd ~
$ pip install M2Crypto -t ~/forlambda/venv/lib/python3.6/site-packages/

$ cd ~/forlambda/venv/lib/python3.6/site-packages/
$ (create python function that uses M2Crypto)
$ zip -r9 ~/forlambda/archive.zip .

Then added to the zip file

/usr/bin/openssl
/usr/lib64/libcrypto.so.10
/usr/lib64/libssl.so.10

And uploaded to Lambda, which is where I am now stuck.

Do I need to do something to get Lambda to use the version of libcrypto.so.10 that I have included in the uploaded zip?

My function:

"""
Wrapper for M2Crypto
https://github.com/mcepl/M2Crypto
https://pypi.org/project/M2Crypto/
"""

from __future__ import print_function
from M2Crypto import RSA
import base64
import json

def decrypt_string(string_b64):
    rsa = RSA.load_key('private_key.pem')
    string_encrypted = base64.b64decode(string_b64)
    bytes = rsa.private_decrypt(string_encrypted, 1)
    string_plaintext = bytes.decode("utf-8")

    response = {
        's': string_plaintext,
        'status': "OK",
        'statuscode': 200
    };
    return response


def lambda_handler(event, context):

    response = ""
    action = event['action']

    if action == "decrypt":
        string_b64 = event['s']
        response = decrypt_string(string_b64)

    return response

Solution

AWS support provided a resolution, upgrading to use Python 3.7 where the issue is resolved:

Our internal team has confirmed that the issue is with Lambda's Python runtime. In a few rare cases, when the Lambda function is being initialised, Lambda is unable to link against the correct OpenSSL libraries - instead linking against Lambda's own in-built OpenSSL binaries.

The team suggests trying this out in the Python3.7 environment where this behaviour has been fixed. Also, python3.7 is compiled with the newer openssl 1.0.2 and you should not have to include the binaries in the Lambda package. ... still had to include the OpenSSL binaries in the package and could not get it working with the default libraries.