Search code examples
c#cffmpegautogen

How to handle FFmpeg hardware decoder initialisation failure

I'm using the FFmpege.AutoGen library in my project and found a crash on a video using hardware decoding. So I went back to the example and tested it there and found the same result.

The example reports the following

Current directory: D:\...\Ruslan-B\FFmpeg.AutoGen\FFmpeg.AutoGen.Example\bin\Debug\net6.0
Running in 64-bit mode.
FFmpeg binaries found in: D:\...\Ruslan-B\FFmpeg.AutoGen\FFmpeg\bin\x64
FFmpeg version info: 5.0-full_build-www.gyan.dev
Use hardware acceleration for decoding?[n]
y
Select hardware decoder:
1. AV_HWDEVICE_TYPE_CUDA
2. AV_HWDEVICE_TYPE_DXVA2
3. AV_HWDEVICE_TYPE_QSV
4. AV_HWDEVICE_TYPE_D3D11VA
5. AV_HWDEVICE_TYPE_OPENCL
6. AV_HWDEVICE_TYPE_VULKAN
Selected [2]

Decoding...
[mpegts @ 0000012917fced00] parser not found for codec klv, packets or times may be invalid.
[h264 @ 000001293b424080] Reinit context to 1920x1088, pix_fmt: yuv420p
[mpegts @ 0000012917fced00] parser not found for codec klv, packets or times may be invalid.
[AVHWDeviceContext @ 000001293b9a6940] Using D3D9Ex device.
codec name: h264
[h264 @ 000001293b8a5040] Decoder GUIDs reported as supported:
[h264 @ 000001293b8a5040] {ee27417f-5e28-4e65-beea-1d26b508adc9}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {bf22ad00-03ea-4690-8077-473346209b7e}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {e07ec519-e651-4cd6-ac84-1370cceec851}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {bcc5db6d-a2b6-4af0-ace4-adb1f787bc89}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81bea4-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81be68-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {d79be8da-0cf1-4c81-b82a-69a4e236f43d}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {f9aaccbb-c2b6-4cfc-8779-5707b1760552}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {705b9d82-76cf-49d6-b7e6-ac8872db013c}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {c528916c-c0af-4645-8cb2-372b6d4adc2a}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {07cfaffb-5a2e-4b99-b62a-e4ca53b6d5aa}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {91cd2d6e-897b-4fa1-b0d7-51dc88010e0a}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {90b899ea-3a62-4705-88b3-8df04b2744e7}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {442b942a-b4d9-4940-bc45-a882e5f919f3}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {8c56eb1e-2b47-466f-8d33-7dbcd63f3df2}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {5b11d51b-2f4c-4452-bcc3-09f2a1160cc0}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {75fc75f7-c589-4a07-a25b-72e03b0383b3}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {107af0e0-ef1a-4d19-aba8-67a163073d13}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {463707f8-a1d0-4585-876d-83aa6d60b89e}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {a4c749ef-6ecf-48aa-8448-50a7a1165ff7}[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {76988a52-df13-419a-8e64-ffcf4a336cf5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {80a3a7bd-89d8-4497-a2b8-2126af7e6eb8}[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {a74ccae2-f466-45ae-86f5-ab8be8af8483}[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81be94-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81bea2-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {49761bec-4b63-4349-a5ff-87ffdf088466}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] No decoder device for codec found
[h264 @ 000001293b8a5040] Failed setup for format dxva2_vld: hwaccel initialisation returned error.
[h264 @ 000001293b8a5040] Reinit context to 1920x1088, pix_fmt: yuv420p
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at FFmpeg.AutoGen.ffmpeg+<>c.<.cctor>b__7_466(FFmpeg.AutoGen.AVFrame*, FFmpeg.AutoGen.AVFrame*, Int32)
   at FFmpeg.AutoGen.ffmpeg.av_hwframe_transfer_data(FFmpeg.AutoGen.AVFrame*, FFmpeg.AutoGen.AVFrame*, Int32)
   at FFmpeg.AutoGen.Example.VideoStreamDecoder.TryDecodeNextFrame(FFmpeg.AutoGen.AVFrame ByRef)
   at FFmpeg.AutoGen.Example.Program.DecodeAllFramesToImages(FFmpeg.AutoGen.AVHWDeviceType)
   at FFmpeg.AutoGen.Example.Program.Main(System.String[])

The errors/messages

...
[h264 @ 0000013533bfb040] No decoder device for codec found
[h264 @ 0000013533bfb040] Failed setup for format dxva2_vld: hwaccel initialisation returned error.
[h264 @ 0000013533bfb040] Reinit context to 1920x1088, pix_fmt: yuv420p

are reported in the call to

error = ffmpeg.avcodec_receive_frame(_pCodecContext, _pFrame);

The result (error) of ffmpeg.avcodec_receive_frame is 0. so there's no obvious error. But the messages make it obviously that the hardware decoder can't handle the video.

The exception is thrown in the call to ffmpeg.av_hwframe_transfer_data()

if (_pCodecContext->hw_device_ctx != null)
{
    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _pFrame, 0).ThrowExceptionIfError();
    frame = *_receivedFrame;
}

My question is how can I determine this beforehand and not use the hardware decoder? i.e. fall back to the software decoder.

Edit: I've found that _pCodecContext->hwaccel is null after the call to ffmpeg.avcodec_receive_frame(_pCodecContext, _pFrame); when for the Truck.ts video but for videos where the hardware decoder is working _pCodecContext->hwaccel is non-null.

Updating the code to

if (_pCodecContext->hw_device_ctx != null && _pCodecContext->hwaccel != null)
{
    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _pFrame, 0).ThrowExceptionIfError();
    frame = *_receivedFrame;
}

Seems to be enough to catch the error. Is the right way to check that the hardware decoder is working or is there a better approach?

Solution

  • You must check _pFrame->hw_frames_ctx are not null.
    If HW failed, hw_frames_ctx in frame will be null