Package files into specific folder of application bundle when deploying to AWS Lambda via Serverless Framework

Question

Context

I am using the aws-node-typescript example of the Serverless Framework. My goal is to integrate Prisma into it.

So far, I have:

1. Created the project locally using serverless create
2. Set up a PostgreSQL database on Railway
3. Installed prisma, ran prisma init, created a basic User model and ran prisma migrate dev successfully
4. Created a second users function by copying the existing hello function
5. Deployed the function using serverless deploy
6. Now in my function, when I instantiate PrismaClient, I get an internal server error and the function logs this error: "ENOENT: no such file or directory, open '/var/task/src/functions/users/schema.prisma'"

My project structure looks as follows:

.
├── README.md
├── package-lock.json
├── package.json
├── prisma
│   ├── migrations
│   │   ├── 20221006113352_init
│   │   │   └── migration.sql
│   │   └── migration_lock.toml
│   └── schema.prisma
├── serverless.ts
├── src
│   ├── functions
│   │   ├── hello
│   │   │   ├── handler.ts
│   │   │   ├── index.ts
│   │   │   ├── mock.json
│   │   │   └── schema.ts
│   │   ├── index.ts
│   │   └── users
│   │       ├── handler.ts
│   │       └── index.ts
│   └── libs
│       ├── api-gateway.ts
│       ├── handler-resolver.ts
│       └── lambda.ts
├── tsconfig.json
└── tsconfig.paths.json

Also, here's the handler for the users function: ts

import { formatJSONResponse } from '@libs/api-gateway';
import { middyfy } from '@libs/lambda';
import { PrismaClient } from '@prisma/client'

const users = async (event) => {

  console.log(`Instantiating PrismaClient inside handler ...`)
  const prisma = new PrismaClient()

  return formatJSONResponse({
    message: `Hello, ${event.queryStringParameters.name || 'there'} welcome to the exciting Serverless world!`,
    event,
  });
};

export const main = middyfy(users);

The problem arises because in order to instantiate PrismaClient, the schema.prisma file needs to be part of the application bundle. Specifically, it needs to be in /var/task/src/functions/users/ as indicated by the error message.

I already adjusted the package.patterns option in my serverless.ts file to look as follows:

package: { individually: true, patterns: ["**/*.prisma"] },

Question

This way, the bundle that's uploaded to AWS Lambda includes the prisma directory in its root, here's the .serverless folder after I ran sls package (I've unzipped users.zip here so that you can see its contents):

.
├── cloudformation-template-create-stack.json
├── cloudformation-template-update-stack.json
├── hello.zip
├── serverless-state.json
├── users
│   ├── prisma
│   │   └── schema.prisma
│   └── src
│       └── functions
│           └── users
│               ├── handler.js
│               └── handler.js.map
└── users.zip

I can also confirm that the deployed version of my AWS Lambda has the same folder structure.

How can I move the users/prisma/schema.prisma file into users/src/functions/users using the patterns in my serverless.ts file?

Answer 1

I found a (pretty ugly) solution. If anyone can think of a more elegant one, I'm still very open to it and happy to give you the points for a correct answer.

Solving the "ENOENT: no such file or directory, open '/var/task/src/functions/users/schema.prisma'" error

To solve this, I just took a very naive approach and manually copied over the schema.prisma file from the prisma directory into the src/functions/users. Here's the file structure I now had:

.
├── README.md
├── package-lock.json
├── package.json
├── prisma
│   ├── migrations
│   │   ├── 20221006113352_init
│   │   │   └── migration.sql
│   │   └── migration_lock.toml
│   └── schema.prisma
├── serverless.ts
├── src
│   ├── functions
│   │   ├── hello
│   │   │   ├── handler.ts
│   │   │   ├── index.ts
│   │   │   ├── mock.json
│   │   │   └── schema.ts
│   │   ├── index.ts
│   │   └── users
│   │       ├── schema.prisma
│   │       ├── handler.ts
│   │       └── index.ts
│   └── libs
│       ├── api-gateway.ts
│       ├── handler-resolver.ts
│       └── lambda.ts
├── tsconfig.json
└── tsconfig.paths.json

This is obviously a horrible way to solve this, because I now have two Prisma schema files in different locations and have to make sure I always update the one in src/functions/users/schema.prisma after changing the original one in prisma/schema.prisma to keep them in sync.

Once I copied this file and redeployed, the schema.prisma file was in place in the right location in the AWS Lambda and the error went away and PrismaClient could be instantiated.

I then added a simple Prisma Client query into the handler:

const users = async (event) => {
  console.log(`Instantiating PrismaClient inside handler ...`)
  const prisma = new PrismaClient()

  const userCount = await prisma.user.count()
  console.log(`There are ${userCount} users in the database`)
  return formatJSONResponse({
    message: `Hello, ${event.queryStringParameters.name || 'there'} welcome to the exciting Serverless world!`,
    event,
  });
};

export const main = middyfy(users);

... and encountered a new error, this time, about the query engine:

Invalid `prisma.user.count()` invocation:


Query engine library for current platform \"rhel-openssl-1.0.x\" could not be found.
You incorrectly pinned it to rhel-openssl-1.0.x

This probably happens, because you built Prisma Client on a different platform.
(Prisma Client looked in \"/var/task/src/functions/users/libquery_engine-rhel-openssl-1.0.x.so.node\")

Searched Locations:

  /var/task/.prisma/client
  /Users/nikolasburk/prisma/talks/2022/serverless-conf-berlin/aws-node-typescript/node_modules/@prisma/client
  /var/task/src/functions
  /var/task/src/functions/users
  /var/task/prisma
  /tmp/prisma-engines
  /var/task/src/functions/users
  
  
  To solve this problem, add the platform \"rhel-openssl-1.0.x\" to the \"binaryTargets\" attribute in the \"generator\" block in the \"schema.prisma\" file:
generator client {
  provider      = \"prisma-client-js\"
  binaryTargets = [\"native\"]
  }

Then run \"prisma generate\" for your changes to take effect.
Read more about deploying Prisma Client: https://pris.ly/d/client-generator

Solving the Query engine library for current platform \"rhel-openssl-1.0.x\" could not be found. error

I'm familiar enough with Prisma to know that Prisma Client depends on a query engine binary that has to be built specifically for the platform Prisma Client will be running on. This can be configured via the binaryTargets field on the generator block in my Prisma schema. The target for AWS Lamda is rhel-openssl-1.0.x.

So I adjusted the schema.prisma file (in both locations) accordingly:

generator client {
  provider = "prisma-client-js"
  binaryTargets = ["native", "rhel-openssl-1.0.x"]
}

After that, I ran npx prisma generate to update the generated Prisma Client in node_modules.

However, this hadn't resolved the error yet, the problem still was the Prisma Client couldn't find the query engine binary.

So I followed the same approach as for the schema.prisma file when it was missing:

1. I manually copied it into src/functions/users (this time from its location inside node_modules/.prisma/libquery_engine-rhel-openssl-1.0.x.so.node)
2. I added the new path to the package.patterns property in my serverless.ts:

   package: { 
     individually: true, 
     patterns: ["**/*.prisma", "**/libquery_engine-rhel-openssl-1.0.x.so.node"],
   },
   

After I redeployed and tested the function, another error occured:

Invalid `prisma.user.count()` invocation:


error: Environment variable not found: DATABASE_URL.
-->  schema.prisma:11
| 
10 |   provider = \"postgresql\"
11 |   url      = env(\"DATABASE_URL\")
| 

Validation Error Count: 1

Solving the Environment variable not found: DATABASE_URL. error

This time, it was pretty straightforward and I went into the AWS Console at https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/functions/aws-node-typescript-dev-users?tab=configure and added a DATABASE_URL env var via the Console UI, pointing to my Postgres instance on Railway: