Possible ways to control code in DLL

Question

I am working on my university project and stuck with a problem. My task is to write a program and .dll to intercept calls from WinAPI. For example I launch my program and it should inject .dll using SetWindowsHookEx to any process by PID. I successfully made this task for function CreateFile, but I need to realise it for multiple functions and allow user to choose which function to intercept via command line arguments.

In my code, when installing hook I define callback function:

HINSTANCE hinst = LoadLibrary(L"ConsoleApplication1.dll");
HOOKPROC addr = (HOOKPROC)GetProcAddress(hinst, "meconnect");
HHOOK handle = SetWindowsHookEx(WH_KEYBOARD, addr, hinst, threadID);

But there are only three arguments in this callback function for launching CallNextHookEx(NULL, code, wParam, lParam) and it seem we can't change them.

The other problem is that when we inject our .dll to another process we are only able to run tasks from DLL_PROCESS_ATTACH section, so it doesn't link with our program, so we can't pass our arguments.

Maybe the solution is to create a temporary file and write arguments to it and then read it from .dll, but I hope you will help me with more elegant solution. I will appreciate any help.

Answer 1

You can use separate function for every hook for example.

BOOL APIENTRY DllMain( HMODULE hModule,
                       DWORD  ul_reason_for_call,
                       LPVOID lpReserved
                     )
{
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        {
            SetWindowsHookEx(WH_CBT, CBTProc, (HINSTANCE) NULL, GetCurrentThreadId());
            SetWindowsHookEx(WH_CALLWNDPROCRET, CallWndRetProc, (HINSTANCE) NULL, GetCurrentThreadId());
            break;
        }
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}


LRESULT CALLBACK CBTProc(int nCode, WPARAM wParam, LPARAM lParam) 
{ 
    if (nCode < 0)
        return CallNextHookEx(___cbt_message_hookptr, nCode, wParam, lParam); 

    // your code here
    return CallNextHookEx(___cbt_message_hookptr, nCode, wParam, lParam); 
} 

LRESULT CALLBACK CallWndRetProc(int nCode, WPARAM wParam, LPARAM lParam) 
{ 
    if (nCode < 0)
        return CallNextHookEx(___callwndprocret_message_hookptr, nCode, wParam, lParam); 

    // your code here
    return CallNextHookEx(___callwndprocret_message_hookptr, nCode, wParam, lParam); 
} 

For reference your DLL_PROCESS_ATTACH issue, it's common problem with communication between intercepted process and your process.

Best solution in my opinion is using Pipes as inter process communication mechanism, bcs it's simplest and most elegant solution.

My advice is you should start another thread from DLL_PROCESS_ATTACH which will be receiving commands via pipe from main application to hook certain functions and also it will be sending intercepted data over same pipe to main application where you will archive or display this data.

Example of sending message over pipe from intercepted process, assuming that you have something listening for this data on other side:

LPTSTR lpszPipename = TEXT("\\\\.\\pipe\\payload_datapipe");
CHAR chReadBuf[1024];
DWORD cbRead = 0;
BOOL fResult;  

fResult = CallNamedPipe( 
    lpszPipename,          // pipe name 
    _Message,              // message to server 
    strlen(_Message),      // message length 
    chReadBuf,             // buffer to receive reply 
    sizeof(chReadBuf),     // size of read buffer 
    &cbRead,               // number of bytes read 
    NMPWAIT_NOWAIT);//NMPWAIT_WAIT_FOREVER); // wait;-) 

if (!fResult)
{
    return;
}

More information's about Named Pipes you can find at relevant documentation pages. https://msdn.microsoft.com/en-us/library/windows/desktop/aa365592%28v=vs.85%29.aspx