How to use clang and distcc to compile on a slave of a different architecture (e.g. Mac/Linux)

I want to use distcc to compile code from my Mac to a bunch of Linux hosts, but I can't figure out how to make everything "line up". I've successfully used distcc from Mac to Mac, so I have a general idea of how to set things up.

I'm using Clang 4.0 and have it installed and working on both Mac and Linux. The following command compiles fine without the distcc at the beginning, but after adding distcc, I get the following issues:

distcc /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/clang++   -I/usr/local/include -I/Users/xaxxon/v8toolkit/./include -I/Users/xaxxon/v8/include  -stdlib=libc++ -g -Werror=return-type -g   -std=gnu++1z -o CMakeFiles/v8toolkit_static.dir/src/v8toolkit.cpp.o -c /Users/xaxxon/v8toolkit/src/v8toolkit.cpp

I'm not sure what compiler is being picked up on Linux, nor do I know how to find out. It's possible it's picking up GCC instead of Clang.

My first concern is this:

clang: warning: argument unused during compilation: '-stdlib=libc++'

My first error is:

In file included from /Users/xaxxon/v8toolkit/src/v8toolkit.cpp:5:
In file included from /usr/include/assert.h:44:
In file included from /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/stdlib.h:94:
/usr/include/stdlib.h:250:20: error: blocks support disabled - compile with -fblocks or pick a deployment target that supports them
int atexit_b(void (^)(void)) __attribute__((availability(macosx,introduced=10.6)));

The next error I get (which becomes the first error if I manually add -fblocks to the compilation command (which isn't needed on a native Mac build) is:

/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:289:13: error: unknown type name '__type_pack_element'
    typedef __type_pack_element<_Ip, _Types...> type;
            ^
/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:289:32: error: expected member name or ';' after declaration specifiers
    typedef __type_pack_element<_Ip, _Types...> type;
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:356:43: error: use of undeclared identifier '__type_pack_element'
      typename _ApplyFn::template __apply<__type_pack_element<_Idx, _Types...>>...
                                          ^
/Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/../include/c++/v1/__tuple:357:6: error: expected a type
    >;

I don't understand if I'm doing something fundamentally wrong or if there's something small I'm missing which is making the Linux compiler act differently.

I just made sure to have Clang on Linux in the same named directory and now am only getting the -fblocks and unused during compilation -stdlib=libc++ issues.

I can get everything to compile (albeit with warnings), but when it links, I get:

ld: warning: ignoring file CMakeFiles/v8toolkit_shared.dir/src/debugger.cpp.o, file was built for
unsupported file format ( 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 0x00 0x00
0x00 0x00 0x00 0x00 0x00 0x00 ) which is not the architecture being linked
(x86_64): CMakeFiles/v8toolkit_shared.dir/src/debugger.cpp.o

Solution

Adding the -target flag fixes everything! In my case, for Mac OS X v10.11 (El Capitan), the target triple is:

-target x86_64-apple-darwin15.6.0

for a build command of:

distcc /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin/clang++  -Dv8toolkit_shared_EXPORTS -I/usr/local/include -I/Users/xaxxon/v8toolkit/./include -isystem /Users/xaxxon/v8/include  -stdlib=libc++ -g -Werror=return-type -target x86_64-apple-darwin15.6.0 -g -fPIC   -std=gnu++1z -o CMakeFiles/v8toolkit_shared.dir/src/v8toolkit.cpp.o -c /Users/xaxxon/v8toolkit/src/v8toolkit.cpp

You can get the current host's target by typing clang -v:

$ clang -v
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-apple-darwin15.6.0 <<==== THIS LINE HERE
Thread model: posix
InstalledDir: /Users/xaxxon/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin/bin

The following CMake lines will grab (and print) the triple for the current machine:

# Get the target triple for the current host by calling clang -v and then stripping out the Target: value from its output.  CMake regex syntax is quite limited.
execute_process(COMMAND ${CMAKE_CXX_COMPILER} -v ERROR_VARIABLE CLANG_VERSION_INFO)
string(REGEX REPLACE ".*Target:[\r\n\t ]*([^\r\n\t]*).*Thread model.*" "\\1" TARGET_TRIPLE ${CLANG_VERSION_INFO})
message(STATUS "TARGET TRIPLE: '${TARGET_TRIPLE}' END")

Much thanks to @duskwuff and oftc.net #llvm for help figuring this out!