DesktopDuplication API produces black frames while certain applications are in fullscreen mode

Question

I'm building an application that is used for taking and sharing screenshots in real time between multiple clients over network.

I'm using the MS Desktop Duplication API to get the image data and it's working smoothly except in some edge cases.

I have been using four games as test applications in order to test how the screencapture behaves in fullscreen and they are Heroes of the Storm, Rainbow Six Siege, Counter Strike and PlayerUnknown's Battlegrounds.

On my own machine which has a GeForce GTX 1070 graphics card; everything works fine both in and out of fullscreen for all test applications. On two other machines that runs a GeForce GTX 980 however; all test applications except PUBG works. When PUBG is running in fullscreen, my desktop duplication instead produces an all black image and I can't figure out why as the Desktop Duplication Sample works fine for all test machines and test applications.

What I'm doing is basically the same as the sample except I'm extracting the pixel data and creating my own SDL(OpenGL) texture from that data instead of using the acquired ID3D11Texture2D directly.

Why is PUBG in fullscreen on GTX 980 the only test case that fails?

Is there something wrong with the way I'm getting the frame, handling the "DXGI_ERROR_ACCESS_LOST" error or how I'm copying the data from the GPU?

Declarations:

IDXGIOutputDuplication* m_OutputDup = nullptr;
Microsoft::WRL::ComPtr<ID3D11Device> m_Device = nullptr;
ID3D11DeviceContext* m_DeviceContext = nullptr;
D3D11_TEXTURE2D_DESC m_TextureDesc;

Initialization:

bool InitializeScreenCapture()
{
    HRESULT result = E_FAIL;
    if (!m_Device)
    {
        D3D_FEATURE_LEVEL featureLevels = D3D_FEATURE_LEVEL_11_0;
        D3D_FEATURE_LEVEL featureLevel;
        result = D3D11CreateDevice(
            nullptr,
            D3D_DRIVER_TYPE_HARDWARE,
            nullptr,
            0,
            &featureLevels,
            1,
            D3D11_SDK_VERSION,
            &m_Device,
            &featureLevel,
            &m_DeviceContext);
        if (FAILED(result) || !m_Device)
        {
            Log("Failed to create D3DDevice);
            return false;
        }
    }

    // Get DXGI device
    ComPtr<IDXGIDevice> DxgiDevice;
    result = m_Device.As(&DxgiDevice);
    if (FAILED(result))
    {
        Log("Failed to get DXGI device);
        return false;
    }

    // Get DXGI adapter
    ComPtr<IDXGIAdapter> DxgiAdapter;
    result = DxgiDevice->GetParent(__uuidof(IDXGIAdapter), &DxgiAdapter);
    if (FAILED(result))
    {
        Log("Failed to get DXGI adapter);
        return false;
    }

    DxgiDevice.Reset();

    // Get output
    UINT Output = 0;
    ComPtr<IDXGIOutput> DxgiOutput;
    result = DxgiAdapter->EnumOutputs(Output, &DxgiOutput);
    if (FAILED(result))
    {
        Log("Failed to get DXGI output);
        return false;
    }

    DxgiAdapter.Reset();

    ComPtr<IDXGIOutput1> DxgiOutput1;
    result = DxgiOutput.As(&DxgiOutput1);
    if (FAILED(result))
    {
        Log("Failed to get DXGI output1);
        return false;
    }

    DxgiOutput.Reset();

    // Create desktop duplication
    result = DxgiOutput1->DuplicateOutput(m_Device.Get(), &m_OutputDup);
    if (FAILED(result))
    {
        Log("Failed to create output duplication);
        return false;
    }

    DxgiOutput1.Reset();

    DXGI_OUTDUPL_DESC outputDupDesc;
    m_OutputDup->GetDesc(&outputDupDesc);

    // Create CPU access texture description
    m_TextureDesc.Width = outputDupDesc.ModeDesc.Width;
    m_TextureDesc.Height = outputDupDesc.ModeDesc.Height;
    m_TextureDesc.Format = outputDupDesc.ModeDesc.Format;
    m_TextureDesc.ArraySize = 1;
    m_TextureDesc.BindFlags = 0;
    m_TextureDesc.MiscFlags = 0;
    m_TextureDesc.SampleDesc.Count = 1;
    m_TextureDesc.SampleDesc.Quality = 0;
    m_TextureDesc.MipLevels = 1;
    m_TextureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_FLAG::D3D11_CPU_ACCESS_READ;
    m_TextureDesc.Usage = D3D11_USAGE::D3D11_USAGE_STAGING;

    return true;
}

Screen capture:

void TeamSystem::CaptureScreen()
{
    if (!m_ScreenCaptureInitialized)
    {
        Log("Attempted to capture screen without ScreenCapture being initialized");
        return false;
    }

    HRESULT result = E_FAIL;
    DXGI_OUTDUPL_FRAME_INFO frameInfo;
    ComPtr<IDXGIResource> desktopResource = nullptr;
    ID3D11Texture2D* copyTexture = nullptr;
    ComPtr<ID3D11Resource> image;

    int32_t attemptCounter = 0;
    DWORD startTicks = GetTickCount();
    do // Loop until we get a non empty frame
    {
        m_OutputDup->ReleaseFrame();
        result = m_OutputDup->AcquireNextFrame(1000, &frameInfo, &desktopResource);
        if (FAILED(result))
        {
            if (result == DXGI_ERROR_ACCESS_LOST) // Access may be lost when changing from/to fullscreen mode(any application); when this happens we need to reaquirce the outputdup
            {
                m_OutputDup->ReleaseFrame();
                m_OutputDup->Release();
                m_OutputDup = nullptr;
                m_ScreenCaptureInitialized = InitializeScreenCapture();
                if (m_ScreenCaptureInitialized)
                {
                    result = m_OutputDup->AcquireNextFrame(1000, &frameInfo, &desktopResource);
                }
                else
                {
                    Log("Failed to reinitialize screen capture after access was lost");
                    return false;
                }
            }

            if (FAILED(result))
            {
                Log("Failed to acquire next frame);
                return false;
            }
        }
        attemptCounter++;

        if (GetTickCount() - startTicks > 3000)
        {
            Log("Screencapture timed out after " << attemptCounter << " attempts");
            return false;
        }

    } while(frameInfo.TotalMetadataBufferSize <= 0 || frameInfo.LastPresentTime.QuadPart <= 0); // This is how you wait for an image containing image data according to SO (https://stackoverflow.com/questions/49481467/acquirenextframe-not-working-desktop-duplication-api-d3d11)

    Log("ScreenCapture succeeded after " << attemptCounter << " attempt(s)");

    // Query for IDXGIResource interface
    result = desktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void**>(&copyTexture));
    desktopResource->Release();
    desktopResource = nullptr;
    if (FAILED(result))
    {
        Log("Failed to acquire texture from resource);
        m_OutputDup->ReleaseFrame();
        return false;
    }

    // Copy image into a CPU access texture
    ID3D11Texture2D* stagingTexture = nullptr;
    result = m_Device->CreateTexture2D(&m_TextureDesc, nullptr, &stagingTexture);
    if (FAILED(result) || stagingTexture == nullptr)
    {
        Log("Failed to copy image data to access texture);
        m_OutputDup->ReleaseFrame();
        return false;
    }

    D3D11_MAPPED_SUBRESOURCE mappedResource;
    m_DeviceContext->CopyResource(stagingTexture, copyTexture);
    m_DeviceContext->Map(stagingTexture, 0, D3D11_MAP_READ, 0, &mappedResource);
    void* copy = malloc(m_TextureDesc.Width * m_TextureDesc.Height * 4);
    memcpy(copy, mappedResource.pData, m_TextureDesc.Width * m_TextureDesc.Height * 4);
    m_DeviceContext->Unmap(stagingTexture, 0);
    stagingTexture->Release();
    m_OutputDup->ReleaseFrame();

    // Create a new SDL texture from the data in the copy varialbe

    free(copy);
    return true;
}

Some notes:

• I have modified my original code to make it more readable so some cleanup and logging in the error handling is missing.
• None of the error or timeout cases(except DXGI_ERROR_ACCESS_LOST) trigger in any testing scenario.
• The "attemptCounter" never goes above 2 in any testing scenario.
• The test cases are limited since I don't have access to a computer which produces the black image case.

Answer 1

The culprit was CopyResource() and how I created the CPU access texture. CopyResource() returns void and that is why I didn't look into it before; I didn't think it could fail in any significant way since I expected it to return bool or HRESULT if that was the case.

In the documentation of CopyResource() does however disclose a couple of fail cases.

This method is unusual in that it causes the GPU to perform the copy operation (similar to a memcpy by the CPU). As a result, it has a few restrictions designed for improving performance. For instance, the source and destination resources:

• Must be different resources.
• Must be the same type.
• Must have identical dimensions (including width, height, depth, and size as appropriate).
• Must have compatible DXGI formats, which means the formats must be identical or at least from the same type group.
• Can't be currently mapped.

Since the initialization code runs before the test application enters fullscreen, the CPU access texture description is set up using the desktop resolution, format etc. This caused CopyResouce() to fail silently and simply now write anything to stagingTexture in the test cases where a non native resoltuion was used for the test application.

In conclusion; I just moved the m_TextureDescription setup to CaptureScreen() and used the description of copyTexture to get the variables I didn't want to change between the textures.

// Create CPU access texture
D3D11_TEXTURE2D_DESC copyTextureDesc;
copyTexture->GetDesc(&copyTextureDesc);

D3D11_TEXTURE2D_DESC textureDesc;
textureDesc.Width = copyTextureDesc.Width;
textureDesc.Height = copyTextureDesc.Height;
textureDesc.Format = copyTextureDesc.Format;
textureDesc.ArraySize = copyTextureDesc.ArraySize;
textureDesc.BindFlags = 0;
textureDesc.MiscFlags = 0;
textureDesc.SampleDesc = copyTextureDesc.SampleDesc;
textureDesc.MipLevels = copyTextureDesc.MipLevels;
textureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_FLAG::D3D11_CPU_ACCESS_READ;
textureDesc.Usage = D3D11_USAGE::D3D11_USAGE_STAGING;

ID3D11Texture2D* stagingTexture = nullptr;
result = m_Device->CreateTexture2D(&textureDesc, nullptr, &stagingTexture);

While this solved the issues I was having; I still don't know why the reinitialization in the handling of DXGI_ERROR_ACCESS_LOST didn't resolve the issue anyway. Does the DesctopDuplicationDescription not use the same dimensions and format as the copyTexture?

I also don't know why this didn't fail in the same way on computers with newer graphics cards. I did however notice that these machines were able to capture fullscreen applications using a simple BitBlt() of the desktop surface.