Hardware acceleration RPI4 FFmpeg

I'm trying to get to work hardware acceleration on my Raspberry pi 4-64. I'm using FFmpeg and AFAIK hwaccel can be reached by using OpenMAX or V4L2-M2M. After '--enable-omx' and 'enable-omx-rpi' for FFmpeg, build fails with error: 'OMX_Core.h not found'. If I will provide manually omx headers, then it will compile but FFmpeg encoding will fail due to missing libraries: bcm_host.so and libopenmaxil.so

I have tried reverting to userland by DISABLE_VC4GRAPHICS = "1", it produced bcm_host.so, but not libopenmaxil.so. I have tried different combinations of virtual providers and graphics settings but without success. Is it possible to access omx hardware acceleration on RPI4-64?

Steps to reproduce the issue:

1.Download latest Poky distro, meta-openembedded, meta-raspberrypi

2.Enable omx, omx-rpi support for FFmpeg

3.Link headers for FFmpeg

4.Build and try to use h264_omx

How do I get missing library libopenmaxil.so and everything I need for hwaccel?

poky master: commit 5d47cdf448b6cff5bb7cc5b0ba0426b8235ec478

meta-openembedded master: commit daa50331352c1f75da3a8ef6458ae3ddf94ef863

meta-raspberrypi master: commit 8d163dd

BTW, by using V4L2-M2M, I'm getting green shadows on the resulting video. Maybe can someone point me in the right direction?

Solution

You have to provide some extra flags to point ffmpeg to the right header and library locations, both at compile time and at run time.

This is what I used to cross-compile ffmpeg for AArxh64:

./configure \
    --arch="${HOST_ARCH}" \
    --target-os="linux" \
    --prefix="/usr/local" \
    --sysroot="${RPI_SYSROOT}" \
    --enable-cross-compile \
    --cross-prefix="${HOST_TRIPLE}-" \
    --toolchain=hardened \
    --enable-gpl --enable-nonfree \
    --enable-avresample \
    --enable-libvpx --enable-libx264 --enable-libxvid \
    --enable-omx --enable-omx-rpi --enable-mmal --enable-neon \
    --enable-shared \
    --disable-static \
    --disable-doc \
    --extra-cflags="$(pkg-config --cflags mmal) \
                    -I${RPI_SYSROOT}/usr/local/include \
                    -I${RPI_SYSROOT}/opt/vc/include/IL" \
    --extra-ldflags="$(pkg-config --libs-only-L mmal) \
                     -Wl,-rpath-link,${RPI_SYSROOT}/opt/vc/lib \
                     -Wl,-rpath,/opt/vc/lib"

Note that pkg-config is configured for cross-compilation as well, it looks in the Raspberry Pi sysroot, not in the build machine root. This is done by setting the right environment variables here.

The -I flags specify the include paths, and the -L flags returned by pkg-config --libs-only-L are the library paths. -Wl passes a list of comma separared arguments to the linker. -rpath-link is used to find shared libraries required by other shared libraries at link time, -rpath is used to find the libraries at run time. This is required because the userland libraries are in a nonstandard location, ld will not search in /opt/vc/lib by default.

You can find the toolchains, Dockerfiles and install scripts I used on my GitHub: https://github.com/tttapa/RPi-Cpp-Toolchain/tree/master/toolchain/docker/rpi3/aarch64/aarch64-cross-build

The userland script is here: https://github.com/tttapa/RPi-Cpp-Toolchain/blob/76ac03741bc7b7da106ae89884c7bada96768a07/toolchain/docker/rpi3/aarch64/aarch64-cross-build/install-scripts/userland.sh
And the ffmpeg script is here: https://github.com/tttapa/RPi-Cpp-Toolchain/blob/76ac03741bc7b7da106ae89884c7bada96768a07/toolchain/docker/rpi3/aarch64/aarch64-cross-build/install-scripts/ffmpeg.sh

There's some more documentation about the compilation process and the files used in the repository here (though not specifically about ffmpeg).