I need to stream my desktop to the AWS MediaLive service and, as a requirement, I must include wallclock timecodes in the stream. The AWS support kindly informed me that for h.264 encoded streams, I need to provide timecodes as "pic_timing SEI messages".
I'm streaming with FFmpeg via the RTMP protocol on Windows 10 so, I tried adding the use_wallclock_as_timestamps and copyts flags to my command.
ffmpeg -f gdigrab -framerate 30 -offset_x 0 -offset_y 0 -video_size 1920x1080 -show_region 1 -use_wallclock_as_timestamps 1 -i desktop -vf scale=320:240 -c:v libx264 -c:a aac -profile:v main -level 3.1 -pix_fmt yuv420p -copyts -f flv rtmp://<ip>:1935/<app>/<stream>
However, the timecodes are not picked up by AWS MediaLive.
My questions are:
use_wallclock_as_timestamps flag actually create timecodes as "pic_timing SEI messages" as required?I'm stumped. I couldn't find the answer in the FFmpeg documentation.
Thanks for your help.
For the "pic_timing SEI messages" support, the following encoder wrappers for H.264/AVC have this implemented in ffmpeg:
h264_qsv
h264_vaapi
h264_nvenc
You can confirm support for the same via:
(a). For the h264_vaapi encoder wrapper (Available on Linux only):
ffmpeg -h encoder=h264_vaapi
Encoder h264_vaapi [H.264/AVC (VAAPI)]:
General capabilities: delay hardware
Threading capabilities: none
Supported hardware devices: vaapi
Supported pixel formats: vaapi_vld
h264_vaapi AVOptions:
-low_power <boolean> E..V....... Use low-power encoding mode (only available on some platforms; may not support all encoding features) (default false)
-idr_interval <int> E..V....... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
-b_depth <int> E..V....... Maximum B-frame reference depth (from 1 to INT_MAX) (default 1)
-rc_mode <int> E..V....... Set rate control mode (from 0 to 6) (default auto)
auto 0 E..V....... Choose mode automatically based on other parameters
CQP 1 E..V....... Constant-quality
CBR 2 E..V....... Constant-bitrate
VBR 3 E..V....... Variable-bitrate
ICQ 4 E..V....... Intelligent constant-quality
QVBR 5 E..V....... Quality-defined variable-bitrate
AVBR 6 E..V....... Average variable-bitrate
-qp <int> E..V....... Constant QP (for P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default 0)
-quality <int> E..V....... Set encode quality (trades off against speed, higher is faster) (from -1 to INT_MAX) (default -1)
-coder <int> E..V....... Entropy coder type (from 0 to 1) (default cabac)
cavlc 0 E..V.......
cabac 1 E..V.......
vlc 0 E..V.......
ac 1 E..V.......
-aud <boolean> E..V....... Include AUD (default false)
-sei <flags> E..V....... Set SEI to include (default identifier+timing+recovery_point)
identifier E..V....... Include encoder version identifier
timing E..V....... Include timing parameters (buffering_period and pic_timing)
recovery_point E..V....... Include recovery points where appropriate
-profile <int> E..V....... Set profile (profile_idc and constraint_set*_flag) (from -99 to 65535) (default -99)
constrained_baseline 578 E..V.......
main 77 E..V.......
high 100 E..V.......
-level <int> E..V....... Set level (level_idc) (from -99 to 255) (default -99)
1 10 E..V.......
1.1 11 E..V.......
1.2 12 E..V.......
1.3 13 E..V.......
2 20 E..V.......
2.1 21 E..V.......
2.2 22 E..V.......
3 30 E..V.......
3.1 31 E..V.......
3.2 32 E..V.......
4 40 E..V.......
4.1 41 E..V.......
4.2 42 E..V.......
5 50 E..V.......
5.1 51 E..V.......
5.2 52 E..V.......
6 60 E..V.......
6.1 61 E..V.......
6.2 62 E..V.......
For VAAPI, the private codec option -sei:v +timing must be passed to h264_vaapi.
(b). For the h264_qsv encoder wrapper:
ffmpeg -h encoder=h264_qsv
Encoder h264_qsv [H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration)]:
General capabilities: delay hybrid
Threading capabilities: none
Supported hardware devices: qsv qsv qsv
Supported pixel formats: nv12 p010le qsv
h264_qsv encoder AVOptions:
-async_depth <int> E..V....... Maximum processing parallelism (from 1 to INT_MAX) (default 4)
-avbr_accuracy <int> E..V....... Accuracy of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
-avbr_convergence <int> E..V....... Convergence of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
-preset <int> E..V....... (from 1 to 7) (default medium)
veryfast 7 E..V.......
faster 6 E..V.......
fast 5 E..V.......
medium 4 E..V.......
slow 3 E..V.......
slower 2 E..V.......
veryslow 1 E..V.......
-rdo <int> E..V....... Enable rate distortion optimization (from -1 to 1) (default -1)
-max_frame_size <int> E..V....... Maximum encoded frame size in bytes (from -1 to 65535) (default -1)
-max_slice_size <int> E..V....... Maximum encoded slice size in bytes (from -1 to 65535) (default -1)
-bitrate_limit <int> E..V....... Toggle bitrate limitations (from -1 to 1) (default -1)
-mbbrc <int> E..V....... MB level bitrate control (from -1 to 1) (default -1)
-extbrc <int> E..V....... Extended bitrate control (from -1 to 1) (default -1)
-adaptive_i <int> E..V....... Adaptive I-frame placement (from -1 to 1) (default -1)
-adaptive_b <int> E..V....... Adaptive B-frame placement (from -1 to 1) (default -1)
-b_strategy <int> E..V....... Strategy to choose between I/P/B-frames (from -1 to 1) (default -1)
-forced_idr <boolean> E..V....... Forcing I frames as IDR frames (default false)
-low_power <boolean> E..V....... enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.) (default false)
-cavlc <int> E..V....... Enable CAVLC (from 0 to 1) (default 0)
-idr_interval <int> E..V....... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
-pic_timing_sei <int> E..V....... Insert picture timing SEI with pic_struct_syntax element (from 0 to 1) (default 1)
-single_sei_nal_unit <int> E..V....... Put all the SEI messages into one NALU (from -1 to 1) (default -1)
-max_dec_frame_buffering <int> E..V....... Maximum number of frames buffered in the DPB (from 0 to 65535) (default 0)
-look_ahead <int> E..V....... Use VBR algorithm with look ahead (from 0 to 1) (default 0)
-look_ahead_depth <int> E..V....... Depth of look ahead in number frames (from 0 to 100) (default 0)
-look_ahead_downsampling <int> E..V....... Downscaling factor for the frames saved for the lookahead analysis (from 0 to 3) (default unknown)
unknown 0 E..V.......
auto 0 E..V.......
off 1 E..V.......
2x 2 E..V.......
4x 3 E..V.......
-int_ref_type <int> E..V....... Intra refresh type (from -1 to 65535) (default -1)
none 0 E..V.......
vertical 1 E..V.......
-int_ref_cycle_size <int> E..V....... Number of frames in the intra refresh cycle (from -1 to 65535) (default -1)
-int_ref_qp_delta <int> E..V....... QP difference for the refresh MBs (from -32768 to 32767) (default -32768)
-recovery_point_sei <int> E..V....... Insert recovery point SEI messages (from -1 to 1) (default -1)
-profile <int> E..V....... (from 0 to INT_MAX) (default unknown)
unknown 0 E..V.......
baseline 66 E..V.......
main 77 E..V.......
high 100 E..V.......
-a53cc <int> E..V....... Use A53 Closed Captions (if available) (from 0 to 1) (default 1)
-aud <int> E..V....... Insert the Access Unit Delimiter NAL (from 0 to 1) (default 0)
-mfmode <int> E..V....... Multi-Frame Mode (from 0 to 2) (default auto)
off 1 E..V.......
auto 2 E..V.......
-repeat_pps <boolean> E..V....... repeat pps for every frame (default false)
The QSV encoder wrapper requires the private codec option -pic_timing_sei:v 1 to be set, and this is the default. This can be overridden to disable the same by passing 0 to the aforementioned parameter.
(c). For h264_nvenc, this option is enabled by default.
Update:
On the HEVC side, ffmpeg's hevc_nvenc wrapper supports it through the private codec option -s12m_tc option that must be set to 1 to enable. This is mentioned because of the work-in-progress support for enhanced RTMP in FFmpeg that implements HEVC.