Search code examples
cmakebuildclangllvmninja

How to build LLVM (clang, lld, mlir) (release/16.x)?


First things first; the versions of various tools that I am using:

OS: Ubuntu 20.04.1 LTS
ld: 2.34
gold: GNU gold (GNU Binutils for Ubuntu 2.34) 1.16
gcc/g++: 9.4.0
cmake: 3.26.0
ninja: 1.10.0

Some details about hardware configurations:

memory: 16 GiB
CPU: Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz

I have cloned the release/16.x branch from llvm-project repository. After creating the build directory and moving under it, I am using the following command:

user@system:~/llvm-project/build$ cmake -G "Ninja" -DLLVM_ENABLE_PROJECTS="lld;clang" -DLLVM_TARGETS_TO_BUILD="RISCV" -DCMAKE_BUILD_TYPE="Debug" -DLLVM_ENABLE_ASSERTIONS=On -DLLVM_PARALLEL_COMPILE_JOBS=1 -DLLVM_PARALLEL_LINK_JOBS=1 ../llvm

user@system:~/llvm-project/build$ ninja -j 1

The build of llvm itself proceeds successfully. However, when the linking process for bin/clang-16 begins, the system hangs (apparently uses too much memory), and after sometime I get the following message (snippet re-generated after core llvm build completed):

[0/613] Linking CXX executable bin/clang-16
FAILED: bin/clang-16
: && /usr/bin/c++ -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra
-Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long
-Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move
-Wno-noexcept-type ...
...
...
... lib/libLLVMDemangle.a -lrt -ldl -lpthread -lm && :
collect2: fatal error: ld terminated with signal 9 [killed]
compilation terminated.
ninja: build stopped: subcommand failed.

(I have skipped some lines in between!)

Any help is welcome.

P.S.: Since I will be playing around with the source code of llvm and clang, I DO NOT want a release build. I think the real problem is static linking, which my computer is not able to handle. I went through the answer to Not able to build LLVM from its source code, but am unable to ascertain where and how to set the flags. Further, the answer to Fatal Error building the llvm source code in Ubuntu also suggests a cmake flag. What should be the correct mixture of flags?


Solution

  • Considering the linked posts and the comments, I first installed clang-12 (the version available for Ubuntu 20.04) and then used the following configuration: (ensure that build and installed directories exist under project root)

    user@system:~/llvm-project/build$ cmake -G "Ninja" \
    -DCMAKE_C_COMPILER=$(which clang) \
    -DCMAKE_CXX_COMPILER=$(which clang++) \
    -DLLVM_ENABLE_PROJECTS="clang;lld;mlir" \
    -DLLVM_TARGETS_TO_BUILD="X86;RISCV" \
    -DCMAKE_BUILD_TYPE="RelWithDebInfo" \
    -DLLVM_ENABLE_ASSERTIONS=ON \
    -DLLVM_BUILD_EXAMPLES=ON \
    -DCMAKE_INSTALL_PREFIX=$(pwd)/../installed \
    -DLLVM_PARALLEL_COMPILE_JOBS=1 \
    -DLLVM_PARALLEL_LINK_JOBS=1 \
    -DLLVM_BUILD_LLVM_DYLIB=ON \
    -DLLVM_LINK_LLVM_DYLIB=ON \
    -DBUILD_SHARED_LIBS=OFF \
    -DLLVM_USE_SPLIT_DWARF=ON \
    -DLLVM_USE_LINKER=gold \
    -DLLVM_OPTIMIZED_TABLEGEN=ON \
    -DLLVM_USE_NEWPM=ON \
    -DCMAKE_EXE_LINKER_FLAGS='-Wl,-no-keep-memory,-fuse-ld=gold' \
    -DLLVM_CCACHE_BUILD=ON \
    ../llvm
    

    After the configurations are completed, the build is performed via:

    ninja -j1 all
    

    The build completed successfully (took about 2h 10m). The subsequent install operation is completed via:

    ninja install
    

    The main objective was to replace static linking of the built libraries with their dynamic linking to prevent the out-of-memory problem. This results in the tools being slower than they should be (that is to say, the llvm tools resulting from this build will work slower), but, given hardware constraints, this is the best approach.


    P.S.

    C and C++ compiler path can also be set with:

    env CC=`which clang` CXX=`which clang++` \
    

    OR

    CC='/lib/llvm-12/bin/clang' \
    CXX='/lib/llvm-12/bin/clang++' \
    

    before the cmake command.

    RelWithDebInfo can be replaced with Debug

    If lld linker is available:

    gold can be replaced with lld

    -fuse-ld=gold can be replaced with -fuse-ld=lld

    -DLLVM_TARGETS_TO_BUILD="X86" can be replaced with -DLLVM_TARGETS_TO_BUILD="X86;RISCV"

    Use -DLLVM_BUILD_EXAMPLES=ON if required.