Building a node module for aws lambda

Question

I'm trying to use the Sharp library in AWS Lambda but it requires the module be compiled for the lambda environment. The instructions say to create an ec2 instance and compile it on there - but I noticed that there are a few tools to help with that but they are all at least a year old with no maintenance. Is there a package that comes with Serverless, or something that's considered the standard way now?

I've found these but they are all at least a year old since a commit

https://github.com/node-hocus-pocus/thaumaturgy

https://github.com/Max-Kolodezniy/aws-lambda-build

https://github.com/tomdale/lambda-packager

Maybe there is a directory somewhere where I can just download a precompiled Sharp library for AWS lambda?

Answer 1

I made it work using sharp-0.17.3-aws-linux-x64-node-6.10.1.tar.gz tarball, that was created on AWS EC2 instance running Nodejs 6.10.1. The tarball contains node_modules/ directory with sharp system binaries (libvips library) specific to the Lambda execution environment.

 

Project structure

To avoid conflicts between my local node_modules/ (Nodejs 7.5 on Mac) and node_modules/ inside the tarball (Nodejs 6.10 on Linux), I'm creating my Lambda service under a subdirectory.
Project structure looks as follows:

node_modules/
service/
  node_modules/ <= sharp-0.17.3-aws-linux-x64-node-6.10.1.tar.gz
  utils/
  handler.js
  package.json  <= engines: node 6.10.1
  serverless.yml
src/
jasmine.json
package.json

Most of the dependencies I need are for development and testing purpose. These are maintained inside root package.json file (also includes sharp, but compiled for my Nodejs 7.5 environment, offering to test image manipulations locally).

My service/handler.js and service/utils/ contains ES6 compatible source code with Lambda function handler – it is transpiled from src/ directory.

If I need other dependencies for production (besides sharp), I install them to services/package.json using --prefix option. But not aws-lambda, neither aws-sdk – they are globally installed within Lambda, meaning no need to include them in deployable .zip file.

npm i -S lodash --prefix services/

It ensures installation of lodash version compatible with Lambda environment, because service/package.json defines Nodejs version to rely on:

{
  "private": true,
  "engines": { "node" : "6.10.1" },
  "dependencies": {
    ...
  }
}

However, there's a nuance — other production dependencies doesn't have to be environment dependent. If so, they won't work, because you install them from your local machine, which isn't equal to Lambda's one.

 

Lambda function deployment

Since Lambda requires .zip archive, I compress contents of my service/ directory. And my Lambda functions works. Everything is ES6 compatible, sharp has Lambda environment binaries and my other production dependency versions correlates with Nodejs 6.10.1.

 

---

Additionally, I'd suggest to use Serverless ⚡️ (I use it too). It dramatically simplifies Lambda functions development and deployment.