How do I target the MSS value in a TCP packet using BPF

I am learning BPF and converting some iptables rules to BPF bytcode. I am primarily using the nfbpf_compile application to do this, rather than trying to write C or Assembler. I am having a lot of luck but the syntax of one rule is escaping me.

I'd like to drop packets with the syn flag set that is also missing an MSS value. In iptables the MSS is targetted with --tcp-option 2. I know that MSS is in the TCP options that start at byte 22 of the TCP packet, and MSS is 'kind' 2. I am able to filter the MSS by using tcp[22:2]==$NUMBER in BPF syntax. However, what I want to do is target SYN packets where the MSS is missing entirely.

I have tried every variant of "null" I can think of but am having no luck.

Does anyone know the equivalent of iptables ! --tcp-option 2 in BPF syntax?

An example of something I have tried:

$ ./nfbpf_compile RAW 'tcp[22:2]==0x0'  (I know this won't work..it's an example)
12,48 0 0 0,84 0 0 240,21 0 8 64,48 0 0 9,21 0 6 6,40 0 0 6,69 4 0 8191,177 0 0 0,72 0 0 22,21 0 1 0,6 0 0 65535,6 0 0 0

# iptables -I INPUT -m bpf --bytecode '12,48 0 0 0,84 0 0 240,21 0 8 64,48 0 0 9,21 0 6 6,40 0 0 6,69 4 0 8191,177 0 0 0,72 0 0 22,21 0 1 0,6 0 0 65535,6 0 0 0
' -j DROP

Solution

TL;DR If you know there are only 0 or 1 TCP options, or if you know the MSS option is always the first option, then you can use the following filter:

tcp && (tcp[tcpflags] == tcp-syn) && ((((tcp[12] & 0xf0) >> 2) < 21) || tcp[20] != 2)

If you don't know this (there are several TCP options and the MSS option may be any of them), which is generally the case, I don't think it's possible to express a matching filter with nfbpf_compile's syntax. In that case, I would recommend writing a C program and loading it with -m bpf --object-pinned /path/to/pinned/bpf.

Let me explain the above filter first. You have two cases to match: 1) there is no TCP option or 2) the first TCP option is not the MSS:

tcp[12] & 0xf0 extracts the data offset field from the TCP header, i.e., the number of 32bit word in the TCP header.
(tcp[12] & 0xf0) >> 2 multiplies this by 4 to get the number of bytes.
If you have less than 21 bytes in your TCP header, then you know there are no TCP options.
tcp[20] != 2 checks that the Option-Kind field of the first TCP option (starts at offset 20) is not 2, the Option-Kind for MSS.

Why is the general case harder to match? TCP options have a variable length (depending on their Option-Kind) and there is a variable, bounded number of TCP options. Say you want to extend the above filter to match on the second TCP option. You first need to know where that option starts; the first option has a variable length so this is not a fixed offset.

With cBPF (the BPF bytecode emitted by nfbpf_compile), you might be able to express that by storing the current option offset in the X register and then loading a byte into register A with the 2nd addressing mode (see the Linux documentation, BPF engine and instruction set). However, I do not think you can do this with the limited nfbpf_compile syntax (assuming it's the same syntax as tcpdump's).