do uploaded files to lambda persist?

Question

I have a node.js function where users kan upload a video file. The video file is uploaded to an S3-bucket. I have a lambda function which listens to the bucket for every upload. Every video that gets uploaded, gets converted via ffmpeg and stored into another bucket, and the original file is deleted.

I messed around with ffmpeg in lambda, quiet difficult. I now have it working, but there is one problem: I'm using fluent-ffmpeg, and streaming data in and out from a bucket doesn't work.

This is my code:

promise...
ffmpeg().input(`https://s3.eu-central-1.amazonaws.com/mybucket.mywebsite.com/${key}`)
        .addInputOption('-analyzeduration 100M')
        .addInputOption('-probesize 100M')
        .toFormat('mp4')
        .addOutputOption("-preset veryfast")
        .addOutputOption("-movflags frag_keyframe+empty_moov")
        .addOutputOption("-crf 28")
        .output('/tmp/output.mp4')
        .on('error', (err, stdout, stderr) => {
            console.log(err);
            console.log('Stdout: %o', stdout);
            console.log('Stderr: %o', stderr);
            reject(err);
        })
        .on('end', () => {
          resolve();
        });
.run();

As you can see, as an input file, I directly read the url of the video file. I wanted to create a readstream (I do this with images), but that didn't work.

I temporary store the created video file as /tmp/output.mp4. After the code is done, I do this:

await putObjectToS3(process.env.MY_NEW_BUCKET, newKey, '/tmp/output.mp4', mimeType);

This works.

My question now is: when the lambda function is done, what happens with the file in /tmp/output.mp4? Is it deleted, or is it stored somewhere until I deleted it. I don't need it anymore so I don't want to pay for the storage of it.

Thanks in advance.

Answer 1

AWS Lambda functions run in containers. Sometimes the containers are reused and sometimes they aren't.

If a container is reused, then any files in /tmp/ will remain. This could result in running out of space because there is a default 512MB of space available. Therefore, it's often a good idea to delete temporary files after use.

There is no charge for the actual files saved in /tmp/, but if you request more than the default 512MB of storage in the Lambda function (up to 10GB) you will be charged accordingly.

For a great overview of AWS Lambda containers, have a read of this series: Operating Lambda: Performance optimization – Part 1 | AWS Compute Blog