Search code examples
c++visual-c++dlldllexportname-mangling

Can I change mangled C++ DLL export symbols? (Binary-compatible but Source-incompatible)


Our C++ project (still) uses the option Treat wchar_t as Built-in: No (/Zc:wchar_t-) from back in the days when it was compiled on VS6.

This results in wchar_t being just a typedef for unsigned short (WORD).

We would like to change this, so that wchar_t is handled as a proper built-in type -- this would greatly ease integration of (modern) libs.

Problem is, we do link to a few DLLs that we cannot recompile that expose their strings as unsigned short* and their headers use wchar_t*. This will result in linker errors when wchar_t is a built-in type, because obviously the export symbols are different.

Changing the headers would require adding a sort of casting layer -- I certainly don't want to add casts to all code that calls into the classes in these headers.

Is it possible to fix the DLLs so that their export symbols would "pretend" to export the built-in wchar_tinstead of WORD? After all, the two types are 100% binary compatible in VC++.

Any other ideas?


Solution

  • It is fine, they are binary compatible.

    The only wall you could run into is the name mangling of the exported function. If the DLLs that you cannot change exported these functions using their decorated C++ name then you'll get a fail whale at runtime when the client cannot find the exported name. This will happen when the programmer didn't use extern "C" or didn't use a .def file to rename the exports. Otherwise easy to see with Dumpbin.exe /exports. Fixing that is painful, you'll need to modify the DLL header to change wchar_t to WORD, a macro can do that, and write little adapter functions that take wchar_t and calls the WORD function by casting.

    If these are exported C++ classes then you have a bigger problem. You'll need to create a new import library for these DLLs. Start with the output you get from Dumpbin.exe /exports, giving you the original names. And create a .def file from that, using its foo = bar option to rename a symbol. Create the import library with the lib.exe /def option. Create a little test DLL to figure out exactly how you should rename the mangled name.