I am currently capturing video via a Blackmagic decklink card on macOS. My audio and video are out of sync. The audio is ahead about a second. I suspect the video is slower on account of encoding latency. My solution is to retard the audio using the ffmpeg adelay filter. I originally added a -af "adelay=1000|1000" to my command to delay the audio by 1000ms but I found that this audio filter did nothing. Consequently, I tried to build a complex_filter, but this failed. My command is producing too many streams that ffmpeg can't route them to the proper rtp endpoint. So what is the best way to delay the audio and can I select which streams map to rtp endpoints?
ffmpeg \
-format_code 23ps \
-f decklink \
-i "DeckLink HD Extreme 3" \
-filter_complex "[0:a] adelay=2s|2s [delayed]" \
-map [delayed] -map 0:v \
-r 24 \
-g 1 \
-s 1920x1080 \
-quality realtime \
-speed 8 \
-threads 8 \
-row-mt 1 \
-tile-columns 2 \
-frame-parallel 1 \
-qmin 30 \
-qmax 35 \
-b:v 2000k \
-pix_fmt yuv420p \
-c:v libvpx-vp9 \
-strict experimental \
-an -f rtp rtp://myurl.com:5004?pkt_size=1300 \
-c:a libopus \
-b:a 128k \
-vn -f rtp rtp://myurl.com:5002?pkt_size=1300
adding a full log when running the command with out any delay:
-filter_complex "[0:a] adelay=2s|2s [delayed]" \
-map [delayed] -map 0:v \
ffmpeg version N-97362-g889ad93c88 Copyright (c) 2000-2020 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
configuration: --prefix=/usr/local --pkg-config-flags=--static --extra-cflags='-fno-stack-check -I/Users/admin/Documents/ffmpeg_build/include -I/Users/admin/Documents/BDS/Mac/include' --extra-ldflags=-L/Users/admin/Documents/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/Users/admin/Documents/ffmpeg_build/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-decklink
libavutil 56. 42.102 / 56. 42.102
libavcodec 58. 80.100 / 58. 80.100
libavformat 58. 42.100 / 58. 42.100
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 77.101 / 7. 77.101
libswscale 5. 6.101 / 5. 6.101
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
[decklink @ 0x7fcfb2000000] Found Decklink mode 1920 x 1080 with rate 23.98
[decklink @ 0x7fcfb2000000] Frame received (#2) - No input signal detected - Frames dropped 1
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, decklink, from 'DeckLink HD Extreme 3':
Duration: N/A, start: 0.000000, bitrate: 797002 kb/s
Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Stream #0:1: Video: rawvideo (UYVY / 0x59565955), uyvy422(progressive), 1920x1080, 795466 kb/s, 23.98 tbr, 1000k tbn, 1000k tbc
[decklink @ 0x7fcfb2000000] Frame received (#3) - Input returned - Frames dropped 2
Stream mapping:
Stream #0:1 -> #0:0 (rawvideo (native) -> vp9 (libvpx-vp9))
Stream #0:0 -> #1:0 (pcm_s16le (native) -> opus (libopus))
Press [q] to stop, [?] for help
[libvpx-vp9 @ 0x7fcfb180d200] v1.8.2
Output #0, rtp, to 'rtp://myurl.com.com:5004?pkt_size=1300':
Metadata:
encoder : Lavf58.42.100
Stream #0:0: Video: vp9 (libvpx-vp9), yuv420p, 1920x1080, q=30-35, 2000 kb/s, 24 fps, 90k tbn, 24 tbc
Metadata:
encoder : Lavc58.80.100 libvpx-vp9
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Output #1, rtp, to 'rtp://myrul.com:5002?pkt_size=1300':
Metadata:
encoder : Lavf58.42.100
Stream #1:0: Audio: opus (libopus), 48000 Hz, stereo, s16, 128 kb/s
Metadata:
encoder : Lavc58.80.100 libopus
SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
t=0 0
a=tool:libavformat 58.42.100
m=video 5004 RTP/AVP 96
c=IN IP4 54.183.58.143
b=AS:2000
a=rtpmap:96 VP9/90000
m=audio 5002 RTP/AVP 97
c=IN IP4 54.183.58.143
b=AS:128
a=rtpmap:97 opus/48000/2
a=fmtp:97 sprop-stereo=1
frame= 434 fps= 24 q=0.0 size= 37063kB time=00:00:18.09 bitrate=16780.7kbits/s speed=1.01x
The -map option is positional and belongs to the first output URL immediately after it. So, the delayed audio should be mapped after the first output and before the 2nd output URL.
ffmpeg \
-format_code 23ps \
-f decklink \
-i "DeckLink HD Extreme 3" \
-filter_complex "[0:a] adelay=2s|2s [delayed]" \
-map 0:v \
-r 24 \
-g 1 \
-s 1920x1080 \
-quality realtime \
-speed 8 \
-threads 8 \
-row-mt 1 \
-tile-columns 2 \
-frame-parallel 1 \
-qmin 30 \
-qmax 35 \
-b:v 2000k \
-pix_fmt yuv420p \
-c:v libvpx-vp9 \
-strict experimental \
-an -f rtp rtp://myurl.com:5004?pkt_size=1300 \
-map [delayed]
-c:a libopus \
-b:a 128k \
-vn -f rtp rtp://myurl.com:5002?pkt_size=1300