Unable to install cvxpy into virtualenv for AWS lambda

Question

I am trying to run the cvxpy package in an AWS lambda function. This package isn't in the SDK, so I've read that I'll have to compile the dependencies into a zip, and then upload the zip into the lambda function.

I've done some research and tried out the links below, but when I try to pip install cvxpy I get error messages - I'm on a Windows box, but I know that AWS Lambda runs on Linux.

Appreciate the help!

http://i-systems.github.io/HSE545/machine%20learning%20all/cvxpy_install/CVXPY%2BInstallation%2BGuide%2Bfor%2BWindows.html

https://programwithus.com/learn-to-code/Pip-and-virtualenv-on-Windows/

https://medium.com/@manivannan_data/import-custom-python-packages-on-aws-lambda-function-5fbac36b40f8

https://www.cvxpy.org/install/index.html

Answer 1

For installing cvxpy on windows it requires c++ build tools (please refer: https://buildmedia.readthedocs.org/media/pdf/cvxpy/latest/cvxpy.pdf)

On Windows:

• I created a lambda layer python directory structure python/lib/python3.7/site-packages (refer: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) and installed my pip packages in that site-packages directory.

pip install cvxpy --target python/lib/python3.7/site-packages

• Then, I zipped the python/lib/python3.7/site-packages as cvxpy_layer.zip and uploaded it to an S3 bucket (layer zipped file max limit is only 50 MB https://docs.aws.amazon.com/lambda/latest/dg/limits.html), to attach it to my lambda layers.
• Now, the layer is ready but the lambda is failing to import the packages as they were installed on a windows machine. (refer: AWS Lambda - unable to import module 'lambda_function')

On Linux:

• I created the same directory structure as earlier python/lib/python3.7/site-packages and installed the cvxpy and zipped it as shown below.
• Later I uploaded the zip file to an S3 bucket and created a new lambda layer.
• Attaching that lambda layer to my lambda function, I colud able to resolve the import issues failing earlier and run the basic cvxpy program on lambda.

mkdir -p alley/python/lib/python3.7/site-packages
pip install cvxpy --target alley/python/lib/python3.7/site-packages
cd alley
zip -rqvT cvxpy_layer.zip .

Lambda layer Image:

Lambda function execution: