Search code examples
linuxebpfbpfxdp-bpf

eBPF - Polling/waiting for userspace in XDP

In my eBPF XDP program, I want to receive a packet, send some information about the packet to userspace, and wait for a decision on what do to with the packet.

My code looks like this:

while (packet.decision == NO_DECISION) {
    poll_userspace(packet);
}

return packet.decision;

Of course this code doesn't pass the verifier since the loop is unbound.

A few solutions I considered:

  • Limit the number of iterations and drop the packet if userspace didn't return a decision in that time.
  • Limit the number of iterations and (somehow) schedule the packet for processing again later, if the limit is reached.

Is polling userspace for a decision, a sensible architectural decision for an XDP eBPF program?

If so, how do you implement this kind of polling/waiting in practice, considering the limitations placed by the verifier?

Solution

  • Is polling userspace for a decision, a sensible architectural decision for an XDP eBPF program?

    It is not. Waiting in an XDP program will block all packet processing for that CPU. XDP is all about making very fast decisions on packets at an early stage to avoid memory allocation in the network stack. The kernel does not really have a mechanism for buffering incoming packets at this layer. eBPF programs in general do not get facilities to delay their decision making. The closest thing I can think of is that you can use a TC program in combination with FQ qdisc to apply an artificial delay to egress(leaving a network interface) packets, and even then you do not get a second eBPF invocation when the packet actually leaves.

    If you really absolutely need to do the decision making in userspace, then AF_XDP is your best bet with eBPF. It essentially allows you to selectively put packets in a ring buffer for you to process in userspace. From usespace you can also send a packet out of that NIC spontaneously (without TX'ing back an existing packet). However, you can't PASS the packet so it appears on the network stack like an XDP program would. So only really useful if your device is positioned as some sort of inline network device.

    Another option is DPDK, which takes the concept further, bypassing the kernel for everything, implementing a custom network stack in userspace. But you end up with similar issues where programs on the same system will need to hook into DPDK to get packets instead of the normal kernel stack.