Search code examples
c++dep

How can I create simple C++ code that runs fine with no Data Execution Prevention (DEP) but will crash with DEP on?


While I understand code that is not marked "executable" will trigger a DEP crash, I am trying to understand what type of common coding practices (in legacy Windows apps) would result in this type of crash.


Solution

  • Something like this:

    
    int main()
    {
        char* s = (char*)malloc(1);
        s[0] = '\xC3';
        void (*p)() = (void (*)())(s);
        p();
    }
    

    ATL did this to allocate thunk for WndProc. The purpose of such WndProc thunks is to embed context parameter and use a method for WndProc instead of a function not taking extra context parameter.

    The fix is easy enough, and does not necessarily include removal of dynamic code allocation:

    • one way is as @Remy pointed out allocate using VirtualAlloc and manage rights using VirtualProtect to make sure execution right is there.
    • easier way is to create a heap with HeapCreate and pass HEAP_CREATE_ENABLE_EXECUTE, allocate code on that heap
    • finally, there are ATL thunk helpers, stating from Windows 10, they can help avoiding having code generation implemented in the program or library. Though this will only work for thunks like ATL thunks, and not a generic solution.