Search code examples
node.jsamazon-web-servicesaws-lambdaffmpeg

How can use ffmpeg in AWS Lambda to clip hls trasnsimision to .mp4


The ffmpeg input is through the public url of the bucket, I want to save the output in a tmp.mp4 file and I want to upload this file to the bucket. Locally the ffmpeg command works perfectly, lasting no more than 3 seconds and the output file weighs less than 1MB

export const handler = async (event) => {
    try {
        const { m3u8, offset, duration, signedURL } = event;

        const clipKey = `clip_output.mp4`;
        const clipFilePath = path.join('/tmp', clipKey);

        execSync(`/opt/ffmpeglib/ffmpeg -i ${process.env.URL_CLOUDFLARE}/${m3u8} -ss ${offset} -t ${duration} -c copy -f mp4 ${clipFilePath}`)

        const fileContent = fs.readFileSync(clipFilePath);
        const resSign = await fetch(signedURL, {
            method: "PUT",
            headers: {
                "Content-Type": "application/octet-stream",
            },
            body: fileContent,
        });

        if (!resSign.ok) throw new Error(`Failed to upload file to S3: ${resSign.statusText}`);

        fs.unlinkSync(clipFilePath);

        return {
            statusCode: 200,
            body: JSON.stringify({
                message: 'Clip procesado y subido correctamente',
                clipKey: path.basename(clipFilePath)
            }),
        };
    } catch (error) {
        console.error("Error al procesar el clip:", error);
        return {
            statusCode: 500,
            body: JSON.stringify({ error: error.message }),
        };
    }
};

The error it shows me is the following

 at genericNodeError (node:internal/errors:984:15)
    at wrappedFn (node:internal/errors:538:14)
    at checkExecSyncError (node:child_process:891:11)
    at execSync (node:child_process:963:15)
    at Runtime.handler (file:///var/task/index.js:16:3)
    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1173:29) {
  status: null,
  signal: 'SIGSEGV',
  output: [
    null,
    <Buffer >,
    <Buffer 66 66 6d 70 65 67 20 76 65 72 73 69 6f 6e 20 4e 2d 37 31 30 36 34 2d 67 64 35 65 36 30 33 64 64 63 30 2d 73 74 61 74 69 63 20 68 74 74 70 73 3a 2f 2f ... 1194 more bytes>
  ],
  pid: 13,
  stdout: <Buffer >,
  stderr: <Buffer 66 66 6d 70 65 67 20 76 65 72 73 69 6f 6e 20 4e 2d 37 31 30 36 34 2d 67 64 35 65 36 30 33 64 64 63 30 2d 73 74 61 74 69 63 20 68 74 74 70 73 3a 2f 2f ... 1194 more bytes>
}
ffmpeg version N-71064-gd5e603ddc0-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil      59. 27.100 / 59. 27.100
libavcodec     61.  9.100 / 61.  9.100
libavformat    61.  4.100 / 61.  4.100
libavdevice    61.  2.100 / 61.  2.100
libavfilter    10.  2.102 / 10.  2.102
libswscale      8.  2.100 /  8.  2.100
libswresample   5.  2.100 /  5.  2.100
libpostproc    58.  2.100 / 58.  2.100
END RequestId: 2fc8c51e-66c6-4c74-aa9c-fa10c11207a0
REPORT RequestId: 2fc8c51e-66c6-4c74-aa9c-fa10c11207a0  Duration: 164.82 ms Billed Duration: 165 ms Memory Size: 2048 MB    Max Memory Used: 88 MB  Init Duration: 185.19 ms

Help to resolve it please


Solution

  • There is a limitation when providing input as public URL to ffmpeg.

    As mentioned here ffmpeg/git-readme.txt

    Notes: A limitation of statically linking glibc is the loss of DNS resolution. Installing nscd through your package manager will fix this.

    This means you can no longer use FFMpeg in any scenario where it will make a DNS query when run on Lambda with a static build, such as passing it a presigned URL. One way to work around this is to download the content, then pass it to FFMpeg to work on.

    This is what I did to download file in /tmp directory.

    async function downloadFileFromPresignedUrl(presignedUrl: string, fileName: string): Promise<string | null> {
      try {
    
        // Send a GET request to the presigned URL
        const response = await axios.get(presignedUrl, {
          responseType: 'arraybuffer'
        });
    
        // Construct the full path for the file in /tmp
        const tmpFilePath = path.join('/tmp', fileName);
    
        // Write the content to a file in /tmp
        fs.writeFileSync(tmpFilePath, response.data);
    
        logger.info(`File downloaded successfully to ${tmpFilePath}`);
        return tmpFilePath;
      } catch (error) {
        logger.error('Error downloading file:', error as Error);
        return null;
      }
    }
    

    The used that path as an input to ffmpeg.

    const fileName = objectKey.split('/').pop();
    
    if (fileName)
      await downloadFileFromPresignedUrl(presignedUrl, fileName)
    else
      throw new Error("Invalid file name")
    
    // Create path for temporary screenshot
    const videoFilePath = path.join('/tmp', fileName);
    const screenshotFilePath = path.join('/tmp', `${fileId}.jpeg`);
    const thumbnailS3Key = `drive/${userId}/thumb/${fileId}.jpeg`.toLowerCase();
    
    await new Promise((resolve, reject) => {
      ffmpeg(videoFilePath)
        .inputOptions(['-ss 00:00:01'])
        .outputOptions([
          '-vframes 1',
          '-vf scale=w=300:h=300:force_original_aspect_ratio=decrease,'
          + 'pad=300:300:(ow-iw)/2:(oh-ih)/2'
        ])
        .output(screenshotFilePath)
        .on('end', () => {
          console.log('Screenshot taken successfully');
          resolve(screenshotFilePath);
        })
        .on('error', (err) => {
          console.error('Error taking screenshot:', err);
          reject(err);
        })
        .run();
    });