Why do the hooking libraries work only occasionally for API Calls in Windows 10?

Question

I used multiple Hooking libraries e.g. Microsoft Detours Express, Mhook, etc. to hook NtWriteVirtualMemory API calls. I wrote following code to hook the API:

    #include <string>
    #include <fstream>
    #include <process.h>
    #include <Windows.h>
    #include <detours.h>

    #pragma comment(lib, "detours.lib")

    using namespace std;

    #define MAX_SIZE 10000
    char tempPath[MAX_SIZE];

    typedef LONG(NTAPI * oNtWriteVirtualMemory)(
            IN HANDLE               ProcessHandle,
            IN PVOID                BaseAddress,
            IN PVOID                Buffer,
            IN ULONG                NumberOfBytesToWrite,
            OUT PULONG              NumberOfBytesWritten OPTIONAL);

    oNtWriteVirtualMemory pNtWriteVirtualMemory = (oNtWriteVirtualMemory)GetProcAddress(GetModuleHandle(L"Ntdll.dll"), "NtWriteVirtualMemory");

    BOOL WINAPI MyNtWriteVirtualMemory(
            IN HANDLE               ProcessHandle,
            IN PVOID                BaseAddress,
            IN PVOID                Buffer,
            IN ULONG                NumberOfBytesToWrite,
            OUT PULONG              NumberOfBytesWritten OPTIONAL
        )
        {
           DWORD pidSource = GetCurrentProcessId();
           DWORD pidDestination = GetProcessId(ProcessHandle);
             if (pidSource != pidDestination)
             {
                 FILE * f1 = fopen(tempPath, "a+");
                    fprintf(f1, "inter-process write from %d to %d!\n", pidSource, pidDestination);
                    fclose(f1);
             }

          return pNtWriteVirtualMemory(ProcessHandle, BaseAddress, Buffer,umberOfBytesToWrite, NumberOfBytesWritten OPTIONAL);
        }

  BOOL APIENTRY DllMain(HMODULE hModule,
        DWORD  ul_reason_for_call,
        LPVOID lpReserved
        )
    {
        strcpy(tempPath, "C:\\Windows\\Temp\\log.txt");
        switch (ul_reason_for_call)
        {
        case DLL_PROCESS_ATTACH:

            try
            {
                DisableThreadLibraryCalls(hModule);

                DetourTransactionBegin();
                DetourUpdateThread(GetCurrentThread());
                DetourAttach(&(PVOID&)pNtWriteVirtualMemory, MyNtWriteVirtualMemory);
                DetourTransactionCommit();
            }
            catch (int e)
            { }
            break;
        case DLL_THREAD_ATTACH:
            try
            {
                DisableThreadLibraryCalls(hModule);

                DetourTransactionBegin();
                DetourUpdateThread(GetCurrentThread());
                DetourAttach(&(PVOID&)pNtWriteVirtualMemory, MyNtWriteVirtualMemory);
                DetourTransactionCommit();
            }
            catch (int e)
            {
            }
            break;
        case DLL_PROCESS_DETACH:

            try
            {
                DetourTransactionBegin();
                DetourUpdateThread(GetCurrentThread());
                DetourDetach(&(PVOID&)pNtWriteVirtualMemory, MyNtWriteVirtualMemory);
                DetourTransactionCommit();
            }
            catch (int e)
            {
            }
            break;
        case DLL_THREAD_DETACH:
            try
            {
                DetourTransactionBegin();
                DetourUpdateThread(GetCurrentThread());
                DetourDetach(&(PVOID&)pNtWriteVirtualMemory, MyNtWriteVirtualMemory);
                DetourTransactionCommit();
            }
            catch (int e)
            {
            }
            break;
        }
        return TRUE;
    }

However, it only works in some specific cases. For example, when the API Monitor program writes its monitoring DLL into the target process to monitor the process's behavior, my hooking code works well and hooks the NtWriteVritualMemory call of API Monitor program as usual. Also, the code hooks the API in Visual Studio very well. When the Visual Studio starts to build a project, it calls NtWriteVirtualMemory for some inter-process writes, and I can see the logs of these calls on my temporary log file!

However, when I write my own code or build another one's code (e.g. Reflective DLL Injection) to call the NtWriteVirtualMemory API, the hook does not detect it at all. I use the Visual Studio 2015 to build these codes to either call the API or hook it. Moreover, I use AppInit_DLLs Infrastructure to make my hooking DLL a global user level hook.

Any clue will be appreciated.

Answer 1

Simple: Your DLL is loaded only into process that load user32.dll. Some process do. Other don't. The one you speak of doesn't:

It's not that the hooking doesn't work. Your DLL isn't even loaded.

Also, hooking in thread attach is probably not what you want, and unhooking in thread detach is almost certainly not what you want.