fatal error: asm/rwonce.h: No such file or directory

Question

Sorry, I'm not very familiar with C and kernel programming, just need to write some bindings around syscalls.

I'm trying to compile a program to call Linux syscall directly. I've installed linux headers to /usr/src/kernel-headers-6.5.4_1/. When I'm compiling it using gcc, the file asm/rwonce.h could not be found. I created a minimal reproducible example:

#include <linux/syscalls.h>

int main() {
    return 0;
}

And I'm compiling it using gcc -I/usr/src/kernel-headers-6.5.4_1/include -o test_program test_program.c. The error is:

In file included from /usr/src/kernel-headers-6.5.4_1/include/linux/export.h:5,
                 from /usr/src/kernel-headers-6.5.4_1/include/linux/linkage.h:7,
                 from /usr/src/kernel-headers-6.5.4_1/include/linux/fs.h:5,
                 from /usr/include/linux/aio_abi.h:31,
                 from /usr/src/kernel-headers-6.5.4_1/include/linux/syscalls.h:79,
                 from test_program.c:1:
/usr/src/kernel-headers-6.5.4_1/include/linux/compiler.h:246:10: fatal error: asm/rwonce.h: No such file or directory
  246 | #include <asm/rwonce.h>
      |          ^~~~~~~~~~~~~~
compilation terminated.

I have double checked the file /usr/src/kernel-headers-6.5.4_1/include/linux/syscalls.h exists. gcc works fine with this include:

$ gcc -E -I/usr/src/kernel-headers-6.5.4_1/include - < /dev/null
# 0 "<stdin>"
# 0 "<built-in>"
# 0 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 0 "<command-line>" 2
# 1 "<stdin>"

I tried to add manually /usr/include but it didn't help: gcc -I/usr/src/kernel-headers-6.5.4_1/include -I/usr/include -o test_program test_program.c

I found that asm/rwonce.h is located at /usr/src/kernel-headers-6.5.4_1/arch/x86/include/generated/asm/rwonce.h, but adding it as gcc -I/usr/src/kernel-headers-6.5.4_1/include -I/usr/include -I/usr/src/kernel-headers-6.5.4_1/arch/x86/include/generated -o test_program test_program.c caused much more problems (first it can't find asm/linkage.h header, then it can't match some function signatures).

I'm using Void linux kernel headers package. My current kernel is 6.5.4_1, gcc version is gcc (GCC) 12.2.0.

How to solve this include issue and compile program with kernel headers correctly?

Answer 1

If you want to manually issue a syscall use the syscall library function from unistd.h (you may need to add #define _DEFAULT_SOURCE at the very top of your source code), do not include kernel headers directly e.g. with -I. What you need should already be available without additional compiler flags. Userspace kernel headers are usually installed along with libc development headers. You may have a libc version that is too old.

RE: "I want to call a new cachestat syscall" - if you have a 6.5 kernel for which userspace headers were correctly installed, you should be able to find the definitions for struct cachestat and struct cachestat_range in sys/mman.h or linux/mman.h.

In any case, these structures are pretty simple. You can just re-define them in your source code if other options don't work:

struct cachestat_range {
    uint64_t off;
    uint64_t len;
};

struct cachestat {
    uint64_t nr_cache;
    uint64_t nr_dirty;
    uint64_t nr_writeback;
    uint64_t nr_evicted;
    uint64_t nr_recently_evicted;
};