Search code examples
c++dllname-manglingstdcall

How to automate generating an import library from undecorated stdcall DLL?


The general way to deal with creating LIB from DLL is described in How to make a .lib file when have a .dll file and a header file - still, to create an import library for DLL with undecorated stdcall functions (e.g. core WinAPI DLL's, kernel32.dll etc.), one has to go through a rather lengthy and complicated process (as described e.g. here). For DLL with a lot of functions this process is very time consuming and error prone - it also fails easily in autobuild situations when the original DLL changes (e.g. due to vendor updates).

Is there an efficient way to automate it?


Solution

  • disclaimer: YMMV. All of the behaviour described here is heavily implementation/version/platform dependent, and I'm not even entirely convinced about some of the things I state here. Still, I think it's a nice collection of miscellaneous ideas, methods and insights, most of them actually working nicely.

    the process I describe can be used as a general way to generate proper implibs for DLLs, not only for stdcall unmangled ones; still, they are the toughest nuts to crack, so I reckon they deserve a bit of a special treatment

    We begin by having only an "alien" (no sources, no headers etc.) DLL file. While there are some dedicated GUI tools that can do this (e.g. this), I'll assume the task should be done in a script, so no GUI whatsoever, text shell commands only.

    Note to MinGW users:

    if you're using a new-ish MinGW (some ancient versions don't support it, but all the recent ones do), you usually don't have to create the import LIB to be able to link with the DLL at all! You just have to:

    a) declare the prototypes of your methods (e.g. in .h file), most probably without __declspec(dllimport) - MinGW uses a strange decoration/mangling for them, and (at least on my setup) they won't link due to __imp_ prefix being expected by the linker instead of the regular __imp__ one,

    b) issue your compilation/linking with additional -LDllDir -lDllNameWithoutDllExtension --enable-stdcall-fixup, e.g.

    g++ example.cpp -L. -lkernel32 --enable-stdcallfixup
    

    That does the trick; the application is properly linked against your DLL, even if it's an undecorated stdcall one.


    In some situations, however, you'll be using either another compiler, or will just require the LIB itself for some reason (redirections, avoiding name clashes etc.). The simplest way is to just create a stub DLL, i.e. for every function prototype you have, create an empty function body (note that while void function can have simply {} body, other functions must have at least { return 0; } or equivalent, or else will raise errors), check if it has __stdcall, wrap all of them in extern "C" { /* your prototypes here */ }, then compile it to DLL, passing a DEF with undecorated names as exports (see 1. below for a simple way how to do that) - discard the DLL generated, keep the LIB, you're done. This can be easily done by a script (e.g. by replacing ; with {} and possibly compiling with ignore 'no value returned in non-void function' error setting or parsing the error log to add return 0; on the offending lines). Sadly, this method requires you to have the .h file (or to generate it based on known DLL interface), so sometimes it's impossible to do it this way.

    If you don't have the headers, you'll have to generate all of the intermediate files needed. The process can be split into 3 parts:

    1. generate a stock DEF file from a DLL,
    2. tailor the DEF to make it possible to generate a proper LIB from it (depending on your luck, you might be able to skip this),
    3. generate a LIB file from the final DEF, and make sure that the LIB has a proper mangled/unmangled/decorated/undecorated state set.

    1

    The first task can be done in many ways. The simplest is to use a dedicated tool - e.g. pexports or gendef, both OSS & available in MinGW extension repo (AFAIR not installed in MinGW by default unless you install everything, though); working with dumpbin /exports is a lot more tricky, since it adds comments on redirects etc. There are many other tools that can do that, like expdef (and they are often simple enough to be directly included in a bigger app, see impdef e.g.). Note that both dlltool and nm will usually fail here!

    gendef %1.dll
    

    et voila. Note that while the DEF must have function names with no _ prefix, no mangling etc. if you're dealing with undecorated stdcall DLL, the @size suffix is usually needed, see below.

    2

    The second task is a bit more tricky; the caveat is that linker will require symbol@size symbols in the LIB to link stdcall calls properly, but you'll probably end with a completely undecorated imports using automated tools. gendef can sometimes regenerate that data (YMMV) - if it does, you're all set to call dlltool or implib (see below). If it doesn't, and you don't have neither the prototypes nor the docs from which you could regenerate the prototypes nor a LIB of any other similar release of this particular DLL, I'd say you're mostly out of luck - with stdcall, you'd have to essentially disassemble each method to know what is the total size of the parameters accepted. Sometimes you could guess/calculate that based on the code that uses it (e.g. count the number of PUSHes before the call), but it's IMO impossible to automate it properly - the values need to be put in DEF manually.

    Note that if you have a header but don't want to use the stub method, you can just create a dummy method calling all of the desired imports - the linker will raise errors which will provide you with decorated names containing the @size in them. Alternatively, you can just calculate it from size_t of the arguments etc.

    3

    Simply using dlltool or implib, generate a LIB from the fixed DEF. The general syntax for for example dlltool would be (as used in a batch file)

    dlltool -d %1.def -D %1.dll -l %1.lib
    

    As a result, you get your LIB.


    Final caveats:

    • you'll probably notice that while MinGW linker happily accepts DEF IMPORTS aliases similar to methodname1@size=methodname2, Microsoft treats them just like methodname1@size, not aliasing them; there's no method to alias a decorated name to a undecorated one in MSVC that I know of,
    • declaring your imports cdecl is never the solution - the symbols may match, but the call will fail in all but the most trivial cases,
    • you will probably notice (use dumpbin /exports /header) that in MSVC implib will generate a LIB with e.g. Symbol name : _function@size Name type : name Name : _function@size - that's not what you want; the proper result (achieved by stubbing) is Symbol name : _function@size Name type : undecorate Name : function - to actually achieve the undecoration, you'd have to hexedit the LIB to change that flag (I know of no implib flag nor DEF setting capable of doing that automagically). The LIB file format is actually identic to COFF/PE one, so you can just edit the import header in the file, changing the relevant byte, located at import header of each import, offset 18 - further reading here (human-friendly) & here (hard docs).