Search code examples
ffmpegrenderingvideo-processing

How do websites render custom videos programatically?


I've noticed a number of websites such as https://biteable.com, https://animoto.com/apps/online-video-maker, etc. that are essentially online video editors that allow users to apply effects, add custom text, etc. and render a video as a result.

I've done a bit of research but can't find any high-level answers on how these websites accomplish creating videos programmatically. I've seen mention of "FFmpeg", but this seems like it would only be able to accomplish basic tasks such as converting a series of images into a video (am I wrong?).

I know very little about video rendering and was hoping someone had an idea to get me started in my research. Are there libraries that exist that can aid to accomplish what these websites are doing? Or are these websites starting from the ground up with lower-level languages to accomplish this?


Solution

  • I've seen mention of "FFmpeg", but this seems like it would only be able to accomplish basic tasks such as converting a series of images into a video (am I wrong?).

    FFmpeg (the ffmpeg command-line tool or usage of its libav* libraries) can do a lot of what the example videos show, but some effects would be difficult if you have no experience using FFmpeg. Knowing how to write arithmetic expressions would be helpful for manipulation of certain filter effects.

    Secondly, ffmpeg is not an editor. It can be used for simplistic editing with some challenges, but it is certainly possible.

    Are there libraries that exist that can aid to accomplish what these websites are doing?

    MLT Framework (melt) is an additional example.

    Or are these websites starting from the ground up with lower-level languages to accomplish this?

    Unknown. I don't think we can answer that. As far as I know they haven't revealed any details of their process, and if someone here does know they probably signed a Non-Disclosure Agreement.

    What their example tells us

    Looking at one of the example videos I can tell that they are using FFmpeg and x264 at least for encoding:

    $ ffmpeg -i Promo-video-created-with-Animoto-online-video-maker-bf8660fd0c.mp4
    …
        encoder         : Lavf57.56.100
    

    Lavf is an alias for libavformat, the FFmpeg muxing/demuxing library. 57.56.100 is the major.minor.micro version number. This particular version indicates that they are probably using the FFmpeg 3.2 release branch for that video.

    $ strings Promo-video-created-with-Animoto-online-video-maker-bf8660fd0c.mp4 | grep x264
    x264 - core 123 r0+5540M 04d6470 - H.264/MPEG-4 AVC codec - Copyright 2003-2012 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0x1:0x111 me=umh subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=8 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=120 keyint_min=1 scenecut=40 intra_refresh=0 rc_lookahead=120 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 vbv_maxrate=5000 vbv_bufsize=10000 crf_max=0.0 nal_hrd=none ip_ratio=1.41 aq=1:1.00
    

    An antiquated x264 was used to create this example, but the example may be just as old. They used -crf 18 -maxrate 5000k -bufsize 10000k