I am trying to build FIO using musl-gcc (we need to use musl due to licensing issues with glibc). I am trying to use the header files provided by musl instead of glibc, but have so far been unsuccessful in compiling FIO. I first ran configure with these options:
my@desktop % ./configure --cc="/usr/local/musl/bin/musl-gcc -I/usr/local/musl/include" --cpu="x86_64" --prefix="/usr/bin"
Operating system Linux
CPU x86_64
Big endian no
Compiler /usr/local/musl/bin/musl-gcc -I/usr/local/musl/include
Cross compile no
Static build no
Wordsize 64
zlib no
Linux AIO support no
POSIX AIO support yes
POSIX AIO support needs -lrt no
POSIX AIO fsync yes
POSIX pshared support yes
Solaris AIO support no
__sync_fetch_and_add yes
__sync_synchronize yes
__sync_val_compare_and_swap yes
libverbs no
rdmacm no
asprintf() yes
vasprintf() yes
Linux fallocate no
POSIX fadvise yes
POSIX fallocate yes
sched_setaffinity(3 arg) yes
sched_setaffinity(2 arg) no
clock_gettime yes
CLOCK_MONOTONIC yes
CLOCK_MONOTONIC_RAW yes
CLOCK_MONOTONIC_PRECISE no
clockid_t yes
gettimeofday yes
fdatasync yes
sync_file_range no
EXT4 move extent yes
Linux splice(2) yes
GUASI no
libnuma no
strsep yes
strcasestr yes
strlcat yes
getopt_long_only() yes
inet_aton yes
socklen_t yes
__thread yes
RUSAGE_THREAD yes
SCHED_IDLE yes
TCP_NODELAY yes
Net engine window_size yes
TCP_MAXSEG yes
RLIMIT_MEMLOCK yes
pwritev/preadv yes
pwritev2/preadv2 no
IPv6 helpers yes
http engine no
Rados engine no
Rados Block Device engine no
setvbuf yes
Gluster API engine no
s390_z196_facilities no
HDFS engine no
MTD no
libpmem no
libpmemblk no
PMDK pmemblk engine no
PMDK dev-dax engine no
PMDK libpmem engine no
DDN's Infinite Memory Engine no
iscsi engine no
lex/yacc for arithmetic no
getmntent yes
getmntinfo no
Static Assert no
bool yes
strndup yes
Valgrind headers no
Zoned block device support no
march_armv8_a_crc_crypto no
cuda no
mkdir(a, b) yes
Build march=native no
CUnit no
__kernel_rwf_t no
-Wimplicit-fallthrough no
MADV_HUGEPAGE yes
gettid no
Consider installing zlib-dev (zlib-devel, some fio features depend on it.
TCMalloc support no
Then ran make:
my@desktop % make
CC crc/crc16.o
CC crc/crc32.o
CC crc/crc32c-arm64.o
In file included from crc/../os/os.h:38,
from crc/crc32c-arm64.c:2:
crc/../os/os-linux.h:16:26: error: linux/unistd.h: No such file or directory
crc/../os/os-linux.h:17:23: error: linux/raw.h: No such file or directory
crc/../os/os-linux.h:18:25: error: linux/major.h: No such file or directory
crc/../os/os-linux.h:19:22: error: linux/fs.h: No such file or directory
In file included from crc/../os/os.h:38,
from crc/crc32c-arm64.c:2:
crc/../os/os-linux.h: In function ‘fio_lookup_raw’:
crc/../os/os-linux.h:175: error: storage size of ‘rq’ isn’t known
crc/../os/os-linux.h:178: error: ‘RAW_MAJOR’ undeclared (first use in this function)
crc/../os/os-linux.h:178: error: (Each undeclared identifier is reported only once
crc/../os/os-linux.h:178: error: for each function it appears in.)
crc/../os/os-linux.h:192: error: ‘RAW_GETBIND’ undeclared (first use in this function)
crc/../os/os-linux.h:175: warning: unused variable ‘rq’
make: *** [crc/crc32c-arm64.o] Error 1
I don't understand why I get the error for this file - it looks to be crc/crc32c-arm64.c, when I'm trying to build for x86_64 (used that in configure option). And it looks for linux/*.h, but there is no 'linux' folder in /usr/local/musl/include. I ran it again by including /usr/include instead (which does have linux/*.h), like so:
my@desktop % ./configure --cc="/usr/local/musl/bin/musl-gcc -I/usr/include" --cpu="x86_64" --prefix="/usr/bin"
but now I get a different error when I run make:
my@desktop % make
CC crc/crc16.o
CC crc/crc32.o
CC crc/crc32c-arm64.o
CC crc/crc32c-intel.o
CC crc/crc32c.o
CC crc/crc64.o
CC crc/crc7.o
CC crc/fnv.o
CC crc/md5.o
CC crc/murmur3.o
CC crc/sha1.o
CC crc/sha256.o
CC crc/sha3.o
CC crc/sha512.o
CC crc/test.o
In file included from /usr/include/stdio.h:75,
from crc/test.c:2:
/usr/include/libio.h:491: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/libio.h:493: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
In file included from crc/test.c:2:
/usr/include/stdio.h:80: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘va_list’
In file included from crc/test.c:2:
/usr/include/stdio.h:371: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:370: warning: conflicting types for built-in function ‘vfprintf’
/usr/include/stdio.h:376: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:376: warning: conflicting types for built-in function ‘vprintf’
/usr/include/stdio.h:379: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:378: warning: conflicting types for built-in function ‘vsprintf’
/usr/include/stdio.h:390: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:389: warning: conflicting types for built-in function ‘vsnprintf’
/usr/include/stdio.h:399: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:417: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:476: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:475: warning: conflicting types for built-in function ‘vfscanf’
/usr/include/stdio.h:483: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:483: warning: conflicting types for built-in function ‘vscanf’
/usr/include/stdio.h:488: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/stdio.h:487: warning: conflicting types for built-in function ‘vsscanf’
In file included from crc/test.c:2:
/usr/include/stdio.h:899: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
In file included from /usr/include/stdio.h:932,
from crc/test.c:2:
/usr/include/bits/stdio2.h:28: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h:26: warning: conflicting types for built-in function ‘__vsprintf_chk’
/usr/include/bits/stdio2.h:44: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h: In function ‘vsprintf’:
/usr/include/bits/stdio2.h:48: error: ‘__ap’ undeclared (first use in this function)
/usr/include/bits/stdio2.h:48: error: (Each undeclared identifier is reported only once
/usr/include/bits/stdio2.h:48: error: for each function it appears in.)
/usr/include/bits/stdio2.h: At top level:
/usr/include/bits/stdio2.h:58: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h:56: warning: conflicting types for built-in function ‘__vsnprintf_chk’
/usr/include/bits/stdio2.h:75: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h: In function ‘vsnprintf’:
/usr/include/bits/stdio2.h:79: error: ‘__ap’ undeclared (first use in this function)
/usr/include/bits/stdio2.h: At top level:
/usr/include/bits/stdio2.h:90: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h:89: warning: conflicting types for built-in function ‘__vfprintf_chk’
/usr/include/bits/stdio2.h:92: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h:91: warning: conflicting types for built-in function ‘__vprintf_chk’
/usr/include/bits/stdio2.h:115: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h: In function ‘vprintf’:
/usr/include/bits/stdio2.h:118: error: ‘__ap’ undeclared (first use in this function)
/usr/include/bits/stdio2.h:118: error: too many arguments to function ‘__vfprintf_chk’
/usr/include/bits/stdio2.h: At top level:
/usr/include/bits/stdio2.h:126: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h: In function ‘vfprintf’:
/usr/include/bits/stdio2.h:128: error: ‘__ap’ undeclared (first use in this function)
/usr/include/bits/stdio2.h:128: error: too many arguments to function ‘__vfprintf_chk’
/usr/include/bits/stdio2.h: At top level:
/usr/include/bits/stdio2.h:137: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h:142: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h:151: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h:196: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h: In function ‘vasprintf’:
/usr/include/bits/stdio2.h:199: error: ‘__ap’ undeclared (first use in this function)
/usr/include/bits/stdio2.h:199: error: too many arguments to function ‘__vasprintf_chk’
/usr/include/bits/stdio2.h: At top level:
/usr/include/bits/stdio2.h:203: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h: In function ‘vdprintf’:
/usr/include/bits/stdio2.h:205: error: ‘__ap’ undeclared (first use in this function)
/usr/include/bits/stdio2.h:205: error: too many arguments to function ‘__vdprintf_chk’
/usr/include/bits/stdio2.h: At top level:
/usr/include/bits/stdio2.h:209: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
/usr/include/bits/stdio2.h: In function ‘obstack_vprintf’:
/usr/include/bits/stdio2.h:213: error: ‘__ap’ undeclared (first use in this function)
/usr/include/bits/stdio2.h:213: error: too many arguments to function ‘__obstack_vprintf_chk’
make: *** [crc/test.o] Error 1
Versions used: musl-1.1.23, fio-3.15, gcc-4.4.6
It builds fine without using musl, if I just use glibc. How can I build FIO successfully using musl instead of glibc? Is this possible?
The whole point of the musl-gcc wrapper script is to invoke gcc with the include and library paths adjusted to isolate it from the host include and library ecosystem (which are assumed to be glibc-based). That includes the kernel headers for your host system. If you want to use any libraries (including "header-only libraries" like the kernel headers) with musl-gcc, you need to build a version against musl instead of glibc and install it in the musl include/library path.
For kernel headers, they don't actually depend on the libc or have any library files; it's just the headers. So you can probably get by with copying (or symlinking) the linux, asm, and asm-generic directories from /usr/include to the musl include dir. Alternatively you can install them from kernel sources.
If you find you need any significant amount of third-party library stuff, though, it makes more sense to just drop musl-gcc and use a real cross-compiler. You can get prebuilt binary ones if you're willing to trust them from musl.cc, or build your own (takes about 15 minutes on a typical system nowadays) with musl-cross-make. This will give you kernel headers automatically, as well as a full set of GCC target libraries that let you build C++ software, OpenMP-using software, etc.