Search code examples
c++cmakearmldbinutils

aarch64 dynamic linker rpath usage with secondary dependency linking

I have two shared libraries: liba and libb, where libb depends on liba and an executable which uses libb. I faced a problem building the project using CMake for aarch64 architecture, while everything works fine with host toolchain. The project looks as following:

├── CMakeLists.txt
├── liba
│   └── main.cpp
├── libb
│   └── main.cpp
└── main
    └── main.cpp

CMakeLists.txt content is the following:

cmake_minimum_required(VERSION 3.15)
project(my_proj CXX)

add_library(a SHARED liba/main.cpp)
set_property(TARGET a PROPERTY LIBRARY_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/a")
add_library(b SHARED libb/main.cpp)
set_property(TARGET b PROPERTY LIBRARY_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/b")
target_link_libraries(b PRIVATE a)

add_executable(main main/main.cpp)
target_link_libraries(main b)

Everything works completely fine with my host g++/ld compiler/linker. However, when I try to build the project using ARM compiler, I get linker error:

$ cmake .. -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++
-- The CXX compiler identification is GNU 9.2.0
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/icblnk/tmp/rm/test/build
$ make
Scanning dependencies of target a
[ 16%] Building CXX object CMakeFiles/a.dir/liba/main.cpp.o
[ 33%] Linking CXX shared library a/liba.so
[ 33%] Built target a
Scanning dependencies of target b
[ 50%] Building CXX object CMakeFiles/b.dir/libb/main.cpp.o
[ 66%] Linking CXX shared library b/libb.so
[ 66%] Built target b
Scanning dependencies of target main
[ 83%] Building CXX object CMakeFiles/main.dir/main/main.cpp.o
[100%] Linking CXX executable main
/usr/lib/gcc/aarch64-linux-gnu/9.2.0/../../../../aarch64-linux-gnu/bin/ld: warning: liba.so, needed by b/libb.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/aarch64-linux-gnu/9.2.0/../../../../aarch64-linux-gnu/bin/ld: b/libb.so: undefined reference to `function_liba_1()'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/main.dir/build.make:85: main] Error 1
make[1]: *** [CMakeFiles/Makefile2:78: CMakeFiles/main.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

The problem is that CMake uses rpath ELF entries to find PRIVATE libraries, which for some reason doesn't work in case of ARM's linker. I tried to look into possible linker options, but failed to find anything useful.

compiler/linker versions I used in the example:

$ ld --version
GNU ld (GNU Binutils) 2.32
$ aarch64-linux-gnu-ld --version
GNU ld (GNU Binutils) 2.32
$ g++ --version
g++ (GCC) 9.2.0
$ aarch64-linux-gnu-g++ --version
aarch64-linux-gnu-g++ (GCC) 9.2.0
Solution

  • Linking fails with cross-compilation as sysroot is implicitly specified so when ld tries to find dependencies using rpath, it also prepends it with sysroot. The issue was fixed in e0f735f8f53543773f01a4f692609d1b23fd3621 and the fix is included since CMake 3.16.0. So now CMake also adds -rpath-link for private dependencies.