I'm trying to find the fastest way to get the complete preprocessed source code (I don't need #line information other comments, just the raw source code) for a C source file.
I have the following little test program which just includes the Windows header file (mini.c):
#include <windows.h>
Using Microsoft Visual Studio 2005, I then run this command:
cl /nologo /P mini.c
This takes 6 seconds to generate a 2.5MB mini.i file; changing this to
cl /nologo /EP mini.c > mini.i
(which skips comments and #line information) needs just 0.5 seconds to write 2.1MB of output.
Is anybody aware of good techniques for improving this even further, without using precompiled headers?
The reason I'm asking is that I wrote a variant of the popular ccache tool for MSVC. As part of the work, the program needs to compute a hash sum of the preprocessed source code (and a few other things). I'd like to make this as fast as possible.
Maybe there is a dedicated preprocessor binary available, or other command line switches which might help?
UPDATE: One idea which just came to my mind: define the WIN32_LEAN_AND_MEAN macro to strip out lots of rarely needed code. This speeds the above preprocessor run up by a factor of approx 3x.
You're repeatedly processing the same source file (<windows.h>) which in turn pulls in a lot of other files. This <windows.h> is located in the default SDK directory.
Now, processing this unchanging file takes serious time. Yet you can rely on it not changing - it's part of the public interface, after all. Hence you could preprocess it - strip out comments, for instance - and pass that version to cl /EP.
Of course, this is typically an I/O bound task, but with a significant CPU part intermixed. An approach which processes multiple sources in parallel will help the total throughput. Measuring single source preprocesing times isn't too relevant.
Finally, measure the time to write the output to NUL. You shouldn't be including the time needed to write mini.i to disk, since you'll intend to pipe the output to md5sum.