Search code examples
c#ffmpegdecodevulkandirectx-11

AVFrame with NV12 format breaks using D3D11 HW acceleration


I have a ffmpeg-autogen video decoder, im trying to use a D3D11 as HW accelerator, but the frame i get in result(with NV12 format) is broken. If im doing the same with Vulkan as accelerator, everything works fine, but vulkan uses more graphic card resources..

For rendering i use OpenTK, received frames i convert to RGB with textures and shaders.

Below my decoder initialization.

public VideoStreamDecoder(AVCodecParameters* parameters, AVCodec* codec)
{
    AVCodec* _codec = codec;
    
    _pCodecContext = ffmpeg.avcodec_alloc_context3(_codec);

    ffmpeg.av_hwdevice_ctx_create(&_pCodecContext->hw_device_ctx, AVHWDeviceType.AV_HWDEVICE_TYPE_D3D11VA, null, null, 0).ThrowExceptionIfError();

    ffmpeg.avcodec_parameters_to_context(_pCodecContext, parameters).ThrowExceptionIfError();
    ffmpeg.avcodec_open2(_pCodecContext, _codec, null).ThrowExceptionIfError();

    CodecName = ffmpeg.avcodec_get_name(_codec->id);
    FrameSize = new Size(_pCodecContext->width, _pCodecContext->height);
    PixelFormat = _pCodecContext->pix_fmt;

    _pFrame = ffmpeg.av_frame_alloc();
    _receivedFrame = ffmpeg.av_frame_alloc();
}

And frames reading function

public bool TryReadNextFrame(out AVFrame frame, AVPacket packet)
{
    int error;

    do
    {
        ffmpeg.avcodec_send_packet(_pCodecContext, &packet).ThrowExceptionIfError();

        error = ffmpeg.avcodec_receive_frame(_pCodecContext, _pFrame);

    } while (error == ffmpeg.AVERROR(ffmpeg.EAGAIN));

    error.ThrowExceptionIfError();

    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _pFrame, 0);
    //here i get NV12 frame

    var clonedFrame = ffmpeg.av_frame_clone(_receivedFrame);

    frame = *clonedFrame;

    return true;
}

And here is result i get with both accelerators:

Vulkan normally working

The same camera, but D3D11 broken

One more intresting moment, is that cameras, that dont work have yuv420p format, the only one working with D3D11 has yuvj420.

I tried changing format using sws_scale from NV12 to NV12, and it worked, but it uses too much CPU. Also tried changin format to rgb before av_hwframe_transfer_data function and rendering it using Vulkan rendering, it sort of works with Vulkan, but with D3D11 it doesnt.

UPDATE

I noticed, that frame linesize was bigger, than width and tried

_receivedFrame->linesize[0] = _receivedFrame->width;

And it helped, now image looks better, but still not perfect.

UPDATE 2

I triead also _receivedFrame->linesize[1] = _receivedFrame->width; and now everything works just as it should, took me whole day to write two lines of code :)


Solution

  • The problem was that frame linesize was actually bigger, than width. Making it equal solved the problem.

    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _pFrame, 0);
    _receivedFrame->linesize[0] = _receivedFrame->width;
    _receivedFrame->linesize[1] = _receivedFrame->width;