Access ingress packet data on BPF_CGROUP_INET_INGRESS hook

Question

I want to inspect the packet data for incoming packets on a specific cgroup. I found the BPF_PROG_TYPE_CGROUP_SKB program type for the BPF_CGROUP_INET_INGRESS attach type can be used to operate over the __sk_buff struct. However, it seems that the __sk_buff struct only exposes the packet headers, and not the data?

For reference, this is my bpf program:

#include <linux/in.h>
#include <linux/tcp.h>

#include <linux/bpf.h>
#include <sys/socket.h>

#include <bpf/bpf_endian.h>
#include <bpf/bpf_helpers.h>

char LICENSE[] SEC("license") = "GPL";

SEC("cgroup_skb/ingress")
int bpf_print_cgroup_skb(struct __sk_buff *skb) {
  long err, data_len;
  char skb_str[100];
  long len = skb->len;

  // Get length of the packet
  if (len > 0) {
    data_len = skb->data_end - skb->data;
    bpf_printk("Packet length: %ld Data length: %ld\n", len, data_len);
    
    if (len > 100) {
      err = bpf_skb_load_bytes(skb, 0, &skb_str, 100);
      if (err < 0) {
        bpf_printk("ERR: bpf_skb_load_bytes failed: %ld\n", err);
        return SK_PASS;
      }
      bpf_printk("Message: %s\n", skb_str);
    }
  }

  return SK_PASS;
}

And this is how I load the program on a cgroup using the userspace program:

static const char *__doc__ = "User space program to load the cgskb bpf program to register sockets\n";

#include <bpf/bpf.h>
#include <bpf/libbpf.h>
#include <errno.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>

#include "../common/common_params.h"
#include "../common/common_user_bpf_xdp.h"

const char *cgroup_dir = "/sys/fs/cgroup/unified";
const char *cgroup_name = "<cgroup_name_here>";

int main(int argc, char **argv) {
  int err, len;
  int cgroup_fd;
  char cgroup_filename[PATH_MAX];
  const char *cgskb_file = "bpf_cgroup_skb.o";
  struct bpf_object *cgskb_obj;

  // Open the cgroup fd -- needed for both attach and detach operations.
  len = snprintf(cgroup_filename, PATH_MAX, "%s/%s", cgroup_dir, cgroup_name);
 
  fprintf(stdout, "Opening cgroup file %s\n", cgroup_filename);
  cgroup_fd = open(cgroup_filename, O_RDONLY);

  // Open, load and attach cgskb_obj if not already attached
  cgskb_obj = bpf_object__open_file(cgskb_file, NULL);
 
  struct bpf_program *cgskb_prog = bpf_object__find_program_by_name(cgskb_obj, "bpf_print_cgroup_skb");
 
  // Load the cgskb program
  err = bpf_object__load(cgskb_obj);
 
  // Attach the cgskb program
  // Using core BPF API as libbpf doesn't support cgskb yet.
  err = bpf_prog_attach(bpf_program__fd(cgskb_prog), cgroup_fd, BPF_CGROUP_INET_INGRESS, 0);
 
  fprintf(stdout, "Successfully loaded BPF program.\n");

  return 0;

exit_cgroup:
  close(cgroup_fd);

fail:
  return -1;
}

Is there any other recommend eBPF program type to get the ingress packet / socket message data.

Answer 1

It indeed seems like reading data other than the headers has been purposefully disallowed. The original commit to add the program type https://github.com/torvalds/linux/commit/0e33661de493db325435d565a4a722120ae4cbf3 states:

This program type is similar to BPF_PROG_TYPE_SOCKET_FILTER, except that it does not allow BPF_LD_[ABS|IND] instructions and hooks up the bpf_skb_load_bytes() helper.

Programs of this type will be attached to cgroups for network filtering and accounting.

So loading packet data via the legacy method (before direct-data access) was blocked at its inception.

A later commit adds direct-data access https://github.com/torvalds/linux/commit/b39b5f411dcfce28ff954e5d6acb2c11be3cb0ec. But its message states:

BPF programs of BPF_PROG_TYPE_CGROUP_SKB need to access headers in the skb. This patch enables direct access of skb for these programs.

Two helper functions bpf_compute_and_save_data_end() and bpf_restore_data_end() are introduced. There are used in __cgroup_bpf_run_filter_skb(), to compute proper data_end for the BPF program, and restore original data afterwards.

And this bpf_compute_and_save_data_end() function limits the data_end to only include the headers, never the body:

cb->data_end = skb->data + skb_headlen(skb);

I have yet to find out what the reason behind this limitation is.