Search code examples
videodirectxms-media-foundationdirect3d11

Does DirectX / Media Foundation have a high-performance intermediate data representation format?

I just used D3D11. I need to intercept the surface in the device / swap X context of thread a and put it into "inter-media-data". This inter-media-data can be locally rendered in the device / swap Y context of another thread B. for performance reasons, I hope that the inter-media-data can copy and operate the original data in the video memory instead of in the system memory as much as possible, and it's better to be independent of these contexts. For example, I hardware encode it as H264 and send it out over the network in another work thread. I tried IMFSample, but I don't know if it can achieve the effect, or if there is a problem with my thinking?

  1. I call MFCreateDxgiSurfaceBuffer and MFCreateSample from the backbuffer of device / swap X to get IMFSample, but I'm not sure whether the sample is related to device / swapx or not? It is hoped that when device / swapx is destroyed, this sample will still be valid.

  2. IMFSample can only query ID3D11Texture2D interface through IMFDXGIBuffer, but in device / swap rendering ID3D11Texture2D, neither CopySubresourceRegion nor CreateRenderTargetView method works. Is there any other efficient and simple rendering method?

Solution

  • There is no (and perhaps no need in) dedicated intermediate format. Media Foundation model is such that it is has a model of samples and buffers which can be implemented with reference to device and API specific data (for example, Direct3D 11). A video frame can be interfaces as IMFSample to Media Foundation, and in the same time have data residing in D3D11 ID3D11Texture2D texture.

    I hope that the inter-media-data can copy and operate the original data in the video memory instead of in the system memory as much as possible

    You just need to keep your data in video memory, in D3D11 textures for that purpose. It is you to provide DXGI device manager to Media Foundation pipeline and so generally it is you who control the lifetime and parameters of hosting D3D11 device. You can have textures (frames) outlive the video encoding step in the case you need the frames for other purposes. You can have this device be shared with swapchain and use textures for both encoding and presentation needs (it's possible to have ID3D11DeviceContext::CopySubresourceRegion working, in particular).