suppress or intercept CRT checks from instances of CRT from other dlls

Question

My program loads several dlls and calls their functions. The dlls can use different versions of CRT.

When C runtime checks the validity of arguments and finds problems, it calls the invalid parameter handle, which in turn, closes the application, with or without the "Send-Don't send" dialog.

I tried calling *_set_invalid_parameter_handler* but it only works if it is called from within the bad dll. I tried SetErrorMode, but all I managed to do is get the process killed without the dialog.

Is there any way to handle those exceptions? I don't care if some resources are compromised. All I want is to allow users to save the configuration. If the dialog appears, they click on it and kill the process.

---

EDIT It turns out the solution to load all versions of CRT or to enumerate all DLLs fails. To make all clear, here is a small example to play with:

This would be my main application (let's call the file application.c):

#include <windows.h>

void myInvalidParameterHandler(const wchar_t* expression, const wchar_t* function, const wchar_t* file, unsigned int line, uintptr_t pReserved) {
   wprintf(L"Invalid parameter detected in function %s. File: %s Line: %d\n", function, file, line);
   wprintf(L"Expression: %s\n", expression);
}

void fixMyProblem() {
}

int main(int argc, char **argv) {
    HMODULE hModule = LoadLibrary("extension.dll");
    void (WINAPI *function)() = GetProcAddress(hModule, "function");
    fixMyProblem();
    function();
}

This application loads a dll that does bad things (it is not developed by me, so I will not accept any solution telling me to fix the bugs there). Lets' call that file extension.c.

#include <stdio.h>

__declspec(dllexport) void function() {
    printf("do bad stuff");
    fopen(NULL, "r");
}

To compile, do:

cl extension.c /link /OUT:extension.dll /DLL
cl application.c

The question is what do I do in function fixMyProblem(), so that I don't get the send/don't send dialog on XP or the application has stopped working dialog on 7.

According to David Gladfelter I should do

void fixMyProblem() {
    _set_invalid_parameter_handler(myInvalidParameterHandler);
}

and also do this for each version CRT available. It turns out that even with one single version of CRT (I use the same for both exe and dll), it still does not work. They both use the same version of CRT, but is seems they do not use the same CRT.

If this is is the case, I assume that the stuff I have to change is inside the DLL. Of course, it does not export *_set_invalid_parameter_handler*.

But to be fair to David Heffernan, here is the implementation for his solution:

#include <Psapi.h>
#pragma comment(lib, "Psapi.lib")
void fixMyProblem() {
    HANDLE hProcess = GetCurrentProcess();
    HMODULE *hModules;
    DWORD requiredSize = 0;
    DWORD secondRequiredSize = 0;
    if (!EnumProcessModules(hProcess, NULL, 0, &requiredSize)) {
        printf("oops\n");
        return;
    }
    hModules = malloc(requiredSize);
    if (EnumProcessModules(hProcess, hModules, requiredSize, &secondRequiredSize)) {
        int i;
        int loadedModules = min(requiredSize, secondRequiredSize) / sizeof(HMODULE);
        for (i = 0; i < loadedModules; i++) {
            void *(WINAPI *_set_invalid_parameter_handler_function)(void *) = (void *(WINAPI *)(void *)) GetProcAddress(hModules[i], "_set_invalid_parameter_handler");
            if (_set_invalid_parameter_handler_function != NULL) {
                _set_invalid_parameter_handler_function(myInvalidParameterHandler);
                printf("fixed dll %d\n", i);
            }
        }
    } else {
        printf("oops\n");
    }
    free(hModules);
}

For my real application, not this test, I get 1 dll fixed (msvcp90.dll). It still does not fix my problem.

I would appreciate any help in solving this.

Answer 1

If the dll is built with a statically linked CRT, then the state and functions of the CRT will be local to that instance of the dll. I am assuming the invalid parameter handler used by the CRT is calling the UnhandledExceptionFilter function, from the OS, to show that "nice" error dialog.

You could try to hook functions like UnhandledExceptionFilter or TerminateProcess, making the dll use your own functions instead. You can do this by parsing the Import Address Table of the loaded dll, search for the function name you are interested, and change the address to point to your function.