BPF_KPROBE macro provides unexpected value of the syscall argument

Question

While playing with libbpf-bootstrap I'm getting unexpected (and strange) function argument for kprobe syscalls. For example for kprobe on close syscall with int close(inf fd) signature, I got enormous fd values like fd=15761240 while expected small int like fd=4. Reproduced this on Debian 11 x64 (kernel 5.10.0-7-amd64) and Ubuntu 21.10 x64 (kernel ~5.13).

Debug code:

#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>

char LICENSE[] SEC("license") = "Dual BSD/GPL";

// accept4 syscall
// int accept4(int sockfd, struct sockaddr *restrict addr, socklen_t *restrict addrlen);
SEC("kretprobe/__x64_sys_accept4")
int BPF_KRETPROBE(accept, int ret) {
    u64 id = bpf_get_current_pid_tgid();
    u32 pid = id >> 32;

    // filter specific pid for simplicity
    if (pid != 31114 || ret < 0) {
        return 0;
    }

    // debug returned file descriptor
    bpf_printk("opened pid=%d fd=%d", pid, ret);
    return 0;
}

// close syscall
// int close(int fd);
SEC("kprobe/__x64_sys_close")
int BPF_KPROBE(close, int fd) {
    u64 id = bpf_get_current_pid_tgid();
    u32 pid = id >> 32;
    // filter specific pid for simplicity
    if (pid != 31114) {
        return 0;
    }

    // debug fd arg (expected to be equal to fd returned on accept4)
    bpf_printk("closed pid=%d fd=%d", pid, fd);
    return 0;
}

Results (see unexpected fd=4 vs fd=15761240 difference):

$ cat /sys/kernel/debug/tracing/trace_pipe
            main-31114   [001] d...  9069.254408: bpf_trace_printk: opened pid=31114 fd=4
            main-31114   [001] d...  9069.321946: bpf_trace_printk: closed pid=31114 fd=15761240

I tried to alter vmlinux.h: at first with vmlinux.h delivered by libbbpf-bootstrap and then with "native" vmlinux.h from the instance OS kernel and on both ways I got the issue above.

Also tried to run the same bpf program in BCC way (compiled with bcc at run-time) with kprobes declared without BPF_KPROBE macro, like that:

int syscall__probe_close_entry(struct pt_regs *ctx, int fd) { ... }

and it worked as expected: fd=4 at all the debug points. Is it a BPF_KPROBE macro bug/incompatibility with the kernel or I'm missing something?

Answer 1

Solved by @anakryiko here

That __x64_sys_close() actually has only one input parameter, and that's struct pt_regs *, which contains all the syscall input arguments. So you have to do something like this to get access to input arguments:

SEC("kprobe/__x64_sys_close")
int BPF_KPROBE(do_sys_close, struct pt_regs *regs)
{
        pid_t pid;
        int fd;

        fd = PT_REGS_PARM1_CORE(regs);

        pid = bpf_get_current_pid_tgid() >> 32;
        bpf_printk("KPROBE ENTRY pid = %d, fd = %d\n", pid, fd);
        return 0;
}

It might be a good idea to add syscall-specific kprobe/kretprobe macros, as this is a common gotcha. Added libbpf/libbpf#425 to keep track of that.