Search code examples
pythondockerffmpegamazon-ecsmoviepy

MoviePY error when running in Docker Container using Amazon Elastic Container Service


I'm developing a script that generates a video with the information provided by the user using Moviepy. The application runs in a docker container.

In my test environment (locally) the application usually runs generating the videos as expected. However, when I deploy to the Amazon Elastic Container Instance environment, the application renders the following error:

/tmp/334.mp31boi_q7b: Invalid data found when processing input
Here are the file infos returned by ffmpeg: 
ffmpeg version 4.2.2-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)

libpostproc 55. 5.100 / 55. 5.100
libswresample 3. 5.100 / 3. 5.100
libswscale 5. 5.100 / 5. 5.100
libavfilter 7. 57.100 / 7. 57.100
libavdevice 58. 8.100 / 58. 8.100
libavformat 58. 29.100 / 58. 29.100
libavcodec 58. 54.100 / 58. 54.100
libavutil 56. 31.100 / 56. 31.100

configuration: 
--enable-gpl 
--enable-version3 
--enable-static 
--disable-debug 
--disable-ffplay 
--disable-indev=sndio 
--disable-outdev=sndio 
--cc=gcc 
--enable-fontconfig 
--enable-frei0r 
--enable-gnutls 
--enable-gmp 
--enable-libgme 
--enable-gray 
--enable-libaom 
--enable-libfribidi 
--enable-libass 
--enable-libvmaf 
--enable-libfreetype 
--enable-libmp3lame 
--enable-libopencore-amrnb 
--enable-libopencore-amrwb 
--enable-libopenjpeg 
--enable-librubberband 
--enable-libsoxr 
--enable-libspeex 
--enable-libsrt 
--enable-libvorbis 
--enable-libopus 
--enable-libtheora 
--enable-libvidstab 
--enable-libvo-amrwbenc 
--enable-libvpx 
--enable-libwebp 
--enable-libx264 
--enable-libx265 
--enable-libxml2 
--enable-libdav1d 
--enable-libxvid 
--enable-libzvbi 
--enable-libzimg

OSError: MoviePy error: failed to read the duration of file /tmp/334.mp31boi_q7b.

To retrieve files sent by the user, I use tempfile.NamedTemporaryFile and manipulate them in memory. I make some slices in the audio to synchronize with images (as if they were slides) and to concatenate them I use the following:

audio_concat = concatenate_audioclips(audio_clips)

and at the end, I generate the video as follows:

with tempfile.NamedTemporaryFile(prefix=video_path, suffix='', delete=False) as data:
    video.write_videofile(data.name, fps=24, codec="libx264", ffmpeg_params=['-f', 'mp4'], threads=multiprocessing.cpu_count())

As said, everything works fine in the container locally, but the same does not happen in ECS.

Does anyone have any idea what I might be doing wrong?

My Dockerfile

FROM ubuntu:latest
ENV DEBIAN_FRONTEND noninteractive

WORKDIR /video_composer

COPY requirements.txt /video_composer/requirements.txt

RUN apt-get update && apt-get install -y software-properties-common gcc && \
    add-apt-repository -y ppa:deadsnakes/ppa

RUN apt-get update && apt-get install -y python3.11 python3-distutils python3-pip python3-apt
RUN apt-get -y update
RUN apt-get install -y ffmpeg

RUN pip install -r /video_composer/requirements.txt

RUN python3 -m certifi

COPY . /video_composer/

EXPOSE 8003

CMD ["uvicorn", "video_composer:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "8003"]

Some solutions consulted and no success

Moviepy unable to read duration of file

https://github.com/Zulko/moviepy/issues/116

When trying to execute the mentioned solutions I continued to have the same error in the ECS but without errors locally.


Solution

  • after extensive code debugging, I have identified the issue. I utilize an asynchronous service that generates audio to be used in the video. Occasionally, the audio generation process took longer, and due to my oversight, I failed to verify before utilizing the audio.

    Therefore, the error OSError: MoviePy error: failed to read the duration of file /tmp/334.mp31boi_q7b. occurs (in my case) because the audio file was empty. That's it!

    When running the audio locally, the latency in the generation process was minimal, which is why no issues occurred locally.