Search code examples
c++overlaydirect3ddirect2ddxgi

ways for Direct2D and Direct3D Interoperability

I want make a Direct2D GUI that will run on a DLL and will render with the Direct3D of the application that I inject into it.

I know that I can simply use ID2D1Factory::CreateDxgiSurfaceRenderTarget to make a DXGI surface and use it as d2d render target, but this require enabling the flag D3D11_CREATE_DEVICE_BGRA_SUPPORT on Direct3D's device.

The problem is that the application creates its device without enabling this flag and, for this reason, ID2D1Factory::CreateDxgiSurfaceRenderTarget fails.

I am trying to find a other way to draw on the application window (externally or inside window's render target) that also works if that window is in full-screen.

I tried these alternatives so far:

  1. Create a d2d render target with ID2D1Factory::CreateDCRenderTarget. This worked, but the part I rendered was blinking/flashing (show and hide very fast in loop). I also called ID2D1DCRenderTarget::BindDC before ID2D1RenderTarget::BeginDraw, but it just blinks but a bit less, so I still had the same issue.

  2. Create a new window that will always be on the top of every other window and render there with d2d but, if the application goes into full-screen, then this window does not show on screen.

  3. Create a second D3D device with enabled the D3D11_CREATE_DEVICE_BGRA_SUPPORT flag and share an ID3D11Texture2D resource between the device of the window and my own, but I wasn't able to make it work... There are not a lot of examples on how to do it. The idea was to create a 2nd device, draw with d2d on that device and then sync the 2 D3D devices – I followed this example (with direct11).

  4. Create a D2D device and share the data of d2d device with d3d device; but, when I call ID2D1Factory1::CreateDevice to create the device it fails because the D3D device is created without enabling the D3D11_CREATE_DEVICE_BGRA_SUPPORT flag. I started with this example.

I've heard of hardware overlay but it works only on some graphics cards and I think I will have problems with this https://learn.microsoft.com/el-gr/windows/win32/medfound/hardware-overlay-support.

I am currently at a dead end; I don't know what to do. Does anyone have any idea that may help me?

Maybe is there any way to draw on screen and work even if a window is in full-screen?

Solution

  • The #3 is the correct one. Here’s a few tips.

    Don’t use keyed mutexes. Don’t use NT handles. The only flag you need is D3D11_RESOURCE_MISC_SHARED.

    To properly synchronize access to the shared texture across devices, use queries. Specifically, you need a query of type D3D11_QUERY_EVENT. The workflow should look like following.

    1. Create a shared texture on one device, open in another one. Doesn’t matter where it’s created and where imported. Don’t forget the D3D11_BIND_RENDER_TARGET flag. Also create a query.

    2. Create D2D device with CreateDxgiSurfaceRenderTarget of the shared texture, render your overlay into the shared texture with D2D and/or DirectWrite.

    3. On the immediate D3D device context with the BGRA flag which you use for D2D rendering, call ID3D11DeviceContext.End once, passing the query. Then wait for the ID3D11DeviceContext.GetData to return S_OK. If you care about electricity/thermals use Sleep(1), or if you prioritize latency, busy wait with _mm_pause() instructions.

    4. Once ID3D11DeviceContext.GetData returned S_OK for that query, the GPU has finished rendering your 2D scene. You can now use that texture on another device to compose into 3D scene.

    The way to compose your 2D content into the render target depends on how do you want to draw your 2D content.

    If that’s a small opaque quad, you can probably CopySubresourceRegion into the render target texture.

    Or, if your 2D content has transparent background, you need a vertex+pixel shaders to render a quad (4 vertices) textured with your shared texture. BTW you don’t necessarily need a vertex/index buffer for that, there’s a well-known trick to do without one. Don’t forget about blend state (you probably want alpha blending), depth/stencil state (you probably want to disable depth test when rendering that quad), also the D3D11_BIND_SHADER_RESOURCE flag for the shared texture.

    P.S. There’s another way. Make sure your code runs in that process before the process created their Direct3D device. Then use something like minhook to intercept the call to D3D11.dll::D3D11CreateDeviceAndSwapChain, in the intercepted function set that BGRA bit you need then call the original function. Slightly less reliable because there’re multiple ways to create a D3D device, but easier to implement, will work faster, and use less memory.