Search code examples
video-streamingintelms-media-foundationintel-media-sdk

How can I use Intel Quicksync for DECODE through media foundation?


Is the way to go to use the Microsoft H.264 MFT? There is a MFT for Quicksync but only for encoding.

The flags on the Microsoft H.264 MFT are Sync and software. I can use a D3D device and use IMFVideoSampleAllocatorEX for some amount of video acceleration, but I was wondering if there was a better way to use Intel Quick Sync through Media Foundation.


Solution

  • Microsoft H.264 Video Encoder is for encoding.

    Intel Quick Sync Video H.264 Encoder MFT is also an encoder object.

    If you are looking for hardware-assisted decoder, the Media Foundation API offers that via H.264 Video Decoder, which is capable of taking advantage of DXVA2 API, which in turn uses hardware capabilities to decode H.264 video.

    UPD. Elaborating follow up questions in the comments below, there is nothing wrong in H.264 Video Decoder MFT being a synchronous MFT. Indeed, hardware MFTs have a documented requirement to be asynchronous. H.264 Video Decoder MFT and more recent H.265/HEVC Video Decoder MFT have a different structure: they combine software decoder as a fallback code path, utilize hardware decoder via internal use of DXVA2 API and are synchronous decoders.

    IHVs do not ship separate hardware MFTs for H.264 and H.265 because the mentioned stock decoder MFTs are already integrating hardware decoding capabilities in efficient way. Dedicated true hardware MFT decoders would not offer any real advantage. For the formats not covered by DXVA2 and/or stock decoders IHVs do ship dedicated decoders in form factor of asynchronous hardware MFTs (M-JPEG, VP8 formats, for example).

    The bottom line is that stock decoders is the suggested method to consume hardware backed decoding. It is possible to use IHV SDK to talk to decoders otherwise. It might offer better flexibility, presumably the complexity is higher esp. in terms of need to integrate with Media Foundation pipeline (if necessary), however the performance gain is not so likely: stock Media Foundation decoders with hardware decoding enabled add reasonably small overhead.