Search code examples
c++buildcompilationcompiler-construction

Idea C++ auto unity build system, would it work?


So this may sound a bit crazy but i had the following idea...

In c++ they call a unity build when you put all your code in one file using #include's or simply copy and pasting. This allows for lightning fast compilation.

Now for simplicity lets assume we have a header only library and we restrict ourselves to only using classes (that way we have a garaunteed namespace). Now i was wondering how well this would work if made automatic. A script/tool would be made to preprocess the .h files; picking out includes and dependencies. The tool would walk every .h file recursively and construct a mapping of dependencies of includes. It would then place these dependencies in one file (main.cpp) using #includes and compile.

Now i want to get your guy's opinions on this idea of automatic unity build creator. Is this smart to do? Would it work as I expect it?


Solution

  • Unity builds are overprized, really.

    This allows for lightning fast compilation.

    This is not quite true these days. Compilation is not I/O bound assuming you have dedicated machine for compilation with enough RAM. Every modern operating system caches files after first read and than get them from memory. You can avoid intermediate files by piping or storing them in RAM drive. So preprocessor work is just concatenating files in most of the time. This is fast. Real time is spent in compilation and linking.

    So any serious application having thousands of files will be huge. Real world example - we work with app where source code takes more than 100MB of data (without system and 3rd party includes). So your system would have to operate on approximate 150MB file. I wouldn't be surprised if some compilers would refuse to compile it. Not to mention time to compile. It is way better to split into smaller files and compile in parallel. Again real example - single thread compile took 40 minutes to finish, running on one server with 16 threads around 2.5-3 minutes and compiling on farm of 16 servers each 16 core with distcc 30 seconds plus some time to link.

    The other benefit of unity builds which is better optimization vanishes when LTO and static build are done.

    Answering your question - yes you can do that. Does it make sense? Doubtful.