I'm having trouble capturing and encoding audio+video on-the-fly on macOS.
I tried two options:
ffmpeg
ffmpeg -threads 0 -f avfoundation -s 1920x1080 -framerate 25 -I 0:0 -async 441 -c:v libx264 -preset medium -pix_fmt yuv420p -crf 22 -c:a libfdk_aac -aq 95 -y
gstreamer
gst-launch-1.0 -ve avfvideosrc device-index=0 ! video/x-raw,width=1920,height=1080,framerate=25/1 ! vtenc_h264 ! queue ! mp4mux name=mux ! filesink location=out.mp4 osxaudiosrc device=0 ! audio/x-raw ! faac midside=false ! queue ! mux.
The ffmpeg option works, but only for lower resolutions. With higher resolutions, the Mac mini (2018 gen) can't do the heavy lifting. I think because I installed ffmpeg with brew, so it wasn't compiled on my machine, meaning it doesn't use the h264 hardware encoder in the Mac?
The gstreamer option works as well, but there's a slight audio/video sync issue (audio is 100ms ahead of the video). I can't seem to add delay to the GStreamer queue (it ignores it):
queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 min-threshold-time=100000000
Anyone who has any experience with this? Thanks!
That change in the queues effects internal flow only. It has no impact on timestamps on the buffers traveling through the pipeline. The timestamps define how the sync between audio and video is.
Try to use the identity element on either the video or audio path and set some timestamp offset via the ts-offset property.