FFMpeg RGB32 to NV12 using SWScale

Question

I'm trying to convert RGB32 frames to NV12 Frames to feed into an encoder.

m_iWidthIn = 1920;
m_iHeightIn = 1080;
m_iWidthOut = (((iWidthIn  + 31) >> 5) << 5) //32bit align
m_heightOut = (((iHeightIn + 31) >> 5) << 5) //32bit align
m_outputPixelFormat = AV_PIX_FMT_NV12;

// allocate and fill buffers

m_sws = ::sws_getContext(m_iWidthIn, m_iHeightIn, AV_PIX_FMT_RGB32, m_iWidthOut, m_iHeightOut, m_outputPixelFormat, SWS_FAST_BILINEAR, nullptr, nullptr, nullptr);
AVFrame* frameOut = av_frame_alloc();
frameOut->height = m_iHeightOut;
frameOut->width = m_iWidthOut;
frameOut->format = m_outputPixelFormat;
av_frame_get_buffer(frameOut, 32);
int linesize[1] = { m_iWidthIn * 4 };
uint8_t * data[1] = { m_inputBuffer  };
if (m_bFlip)
{
    data[0] += linesize[0] * (m_iHeightIn - 1);
    linesize[0] = -linesize[0];
}
::sws_scale(m_sws, data, linesize, 0, m_iHeightIn, frameOut->data, frameOut->linesize);
::av_image_copy_to_buffer(pOutputBuffer, lDataLen, frameOut->data, frameOut->linesize, m_outputPixelFormat, m_iWidthOut, m_iHeightOut, 32);

If I make m_outputPixelFormat AV_PIX_FMT_RGB32 and use a DMO colorspace converter, the video comes out correctly. However if I change it to NV12, I end up with a slanted video with missing data at the bottom. I know this is caused by me copying the data incorrectly out of the buffer, but I'm unsure what I'm doing incorrectly.

Answer 1

Your problem is here:

m_heightOut = (((iHeightIn + 31) >> 5) << 5) //32bit align

You don't need to align height. So frameOut->data has m_iHeightIn height. The correct line is:

m_heightOut = iHeightIn;