Search code examples
winapiopengldetours

Detours crashes with wglMakeCurrent


I am using Detours on my project in order to save the GL Context of a Window.

So I have followed the code sample that comes with Detours 3.0 Express Edition:

static BOOL (WINAPI * trueWglMakeCurrent)(HDC, HGLRC) = wglMakeCurrent;
BOOL WINAPI hookedWglMakeCurrent(HDC hdc, HGLRC hglrc);

BOOL WINAPI hookedWglMakeCurrent(HDC hdc, HGLRC hglrc)
{
    wContext = hglrc;
    wDC = hdc;

    return trueWglMakeCurrent(hdc, hglrc);  //CRASH HERE
    //return TRUE;
}

But it crashes when invoking the original WGL method. If I change that return to return TRUE, it won't crash. But it obviously won't render anything either.

hdc and hglrc have valid addresses and they probably correspond to the GL Context I need, since this is called right after the desired Window creation.

EDIT

It does not crash when applying the same method to other OpenGL functions, e.g. SwapBuffers, glFinish, etc.

In the case of an isolated test in which I simply load my DLL and execute the wglMakeCurrent it works.

In the case of injecting my DLL into an application and then using wglMakeCurrent on this application, Detours causes an infinite recursion on the trueWglMakeCurrent call.


Solution

  • The problem was the infinite recursion that was being caused by not linking the functions properly.

    Detours creates a function pointer to the original wglMakeCurrent, which in my case I stored in trueWglMakeCurrent. However it works only if I declare it and use it on the same file that has the DLLMain and the DetourAPI usage. Exactly like the example:

    static BOOL (WINAPI * trueWglMakeCurrent)(HDC,HGLRC) = wglMakeCurrent;
    
    GTNSPECTRA_C_EXPORT BOOL WINAPI hookedWglMakeCurrent(HDC hDC, HGLRC hRC)
    {
        ...
        return trueWglMakeCurrent(hDC, hRC);
    }
    
    BOOL APIENTRY DllMain( HMODULE hModule,
                       DWORD  reason,
                       LPVOID lpReserved)
    {
        ...
        DetourRestoreAfterWith();
    
        DetourTransactionBegin();
        DetourUpdateThread(GetCurrentThread());
        DetourAttach(&(PVOID&)trueWglMakeCurrent, hookedWglMakeCurrent);
        DetourTransactionCommit();
    
        ...
        return TRUE;
     }