I'm having problems with setting up port forwarding on Asuswrt-Merlin when I'm connected to VPN. It works great when I'm not connected to the VPN, however. The forwarding is supposed to happen outside of the VPN, so all outgoing traffic is via the VPN unless it's through a port opened by the NAT by connecting to my public IP.
This is the result of iptables-save:
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:16:42 2015
*raw
:PREROUTING ACCEPT [90913:23933556]
:OUTPUT ACCEPT [39123:12900614]
-A PREROUTING -s [remote host ip]/32 -j TRACE
COMMIT
# Completed on Mon Jul 6 21:16:42 2015
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:16:42 2015
*nat
:PREROUTING ACCEPT [1743:150138]
:INPUT ACCEPT [135:10064]
:OUTPUT ACCEPT [20:3734]
:POSTROUTING ACCEPT [20:3734]
:DNSFILTER - [0:0]
:LOCALSRV - [0:0]
:PCREDIRECT - [0:0]
:VSERVER - [0:0]
:VUPNP - [0:0]
-A PREROUTING -d [vpn public ip]/32 -j VSERVER
-A PREROUTING -d [public ip]/32 -j VSERVER
-A OUTPUT -s [remote host ip]/32 -j LOG
-A POSTROUTING ! -s [vpn public ip]/32 -o ppp5 -j MASQUERADE
-A POSTROUTING ! -s [public ip]/32 -o eth0 -j MASQUERADE
-A POSTROUTING -m mark --mark 0xb400 -j MASQUERADE
-A VSERVER -p tcp -m tcp --dport 5522 -j DNAT --to-destination 192.168.1.110
-A VSERVER -j VUPNP
COMMIT
# Completed on Mon Jul 6 21:16:42 2015
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:16:42 2015
*mangle
:PREROUTING ACCEPT [233459:124857411]
:INPUT ACCEPT [98539:61619123]
:FORWARD ACCEPT [133882:63069590]
:OUTPUT ACCEPT [82724:24102754]
:POSTROUTING ACCEPT [216675:87184104]
-A PREROUTING -d [public ip]/32 ! -i eth0 -j MARK --set-xmark 0xb400/0xffffffff
COMMIT
# Completed on Mon Jul 6 21:16:42 2015
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:16:42 2015
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [17618:4348249]
:FUPNP - [0:0]
:PControls - [0:0]
:logaccept - [0:0]
:logdrop - [0:0]
-A INPUT -i eth0 -p icmp -m icmp --icmp-type 8 -j DROP
-A INPUT -i eth0 -m state --state INVALID -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -m state --state NEW -j ACCEPT
-A INPUT -i br0 -m state --state NEW -j ACCEPT
-A INPUT -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A INPUT -j DROP
-A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD ! -i br0 -o eth0 -j DROP
-A FORWARD -i eth0 -m state --state INVALID -j DROP
-A FORWARD -i br0 -o br0 -j ACCEPT
-A FORWARD -m conntrack --ctstate DNAT -j ACCEPT
-A FORWARD -i br0 -j ACCEPT
-A FORWARD ! -i br0 -o ppp5 -j DROP
-A PControls -j ACCEPT
-A logaccept -m state --state NEW -j LOG --log-prefix "ACCEPT " --log-tcp-sequence --log-tcp-options --log-ip-options
-A logaccept -j ACCEPT
-A logdrop -m state --state NEW -j LOG --log-prefix "DROP " --log-tcp-sequence --log-tcp-options --log-ip-options
-A logdrop -j DROP
COMMIT
# Completed on Mon Jul 6 21:16:42 2015
The traced result of this is (for two packages in a row):
Jul 6 21:11:14 kernel: TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22744 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12700000000)
Jul 6 21:11:14 kernel: TRACE: mangle:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22744 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12700000000)
Jul 6 21:11:14 kernel: TRACE: nat:PREROUTING:rule:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22744 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12700000000)
Jul 6 21:11:14 kernel: TRACE: nat:VSERVER:rule:1 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22744 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12700000000)
Jul 6 21:11:15 kernel: TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22745 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12900000000)
Jul 6 21:11:15 kernel: TRACE: mangle:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22745 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12900000000)
Jul 6 21:11:15 kernel: TRACE: nat:PREROUTING:rule:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22745 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12900000000)
Jul 6 21:11:15 kernel: TRACE: nat:VSERVER:rule:1 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote host ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22745 DF PROTO=TCP SPT=45584 DPT=5522 SEQ=3817215282 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEF12900000000)
As far as I can read that trace, the packet is simply lost as soon as it reaches the nat:VSERVER:rule:1, which is the rule that executes -j DNAT (as confirmed by counters going up).
In case it is of any use, here is the iptables-save for when I'm not connected to the VPN and the port forwarding is actually working.
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:33:17 2015
*raw
:PREROUTING ACCEPT [238017:110134781]
:OUTPUT ACCEPT [86340:25301671]
-A PREROUTING -s [remote ip]/32 -j TRACE
COMMIT
# Completed on Mon Jul 6 21:33:17 2015
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:33:17 2015
*nat
:PREROUTING ACCEPT [7421:918988]
:INPUT ACCEPT [203:11322]
:OUTPUT ACCEPT [18:2335]
:POSTROUTING ACCEPT [79:14834]
:DNSFILTER - [0:0]
:LOCALSRV - [0:0]
:PCREDIRECT - [0:0]
:VSERVER - [0:0]
:VUPNP - [0:0]
-A PREROUTING -d [public ip]/32 -j VSERVER
-A OUTPUT -s [remote ip]/32 -j LOG
-A POSTROUTING ! -s [public ip]/32 -o eth0 -j MASQUERADE
-A POSTROUTING -m mark --mark 0xb400 -j MASQUERADE
-A VSERVER -p tcp -m tcp --dport 5522 -j DNAT --to-destination 192.168.1.110
-A VSERVER -j VUPNP
COMMIT
# Completed on Mon Jul 6 21:33:17 2015
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:33:17 2015
*mangle
:PREROUTING ACCEPT [380592:211060643]
:INPUT ACCEPT [153369:102799194]
:FORWARD ACCEPT [225946:108037401]
:OUTPUT ACCEPT [129943:36503787]
:POSTROUTING ACCEPT [355967:144555180]
-A PREROUTING -d [public ip]/32 ! -i eth0 -j MARK --set-xmark 0xb400/0xffffffff
COMMIT
# Completed on Mon Jul 6 21:33:17 2015
# Generated by iptables-save v1.4.14 on Mon Jul 6 21:33:17 2015
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [2494:2146701]
:FUPNP - [0:0]
:PControls - [0:0]
:logaccept - [0:0]
:logdrop - [0:0]
-A INPUT -i eth0 -p icmp -m icmp --icmp-type 8 -j DROP
-A INPUT -i eth0 -m state --state INVALID -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -m state --state NEW -j ACCEPT
-A INPUT -i br0 -m state --state NEW -j ACCEPT
-A INPUT -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A INPUT -j DROP
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD ! -i br0 -o eth0 -j DROP
-A FORWARD -i eth0 -m state --state INVALID -j DROP
-A FORWARD -i br0 -o br0 -j ACCEPT
-A FORWARD -m conntrack --ctstate DNAT -j ACCEPT
-A FORWARD -i br0 -j ACCEPT
-A PControls -j ACCEPT
-A logaccept -m state --state NEW -j LOG --log-prefix "ACCEPT " --log-tcp-sequence --log-tcp-options --log-ip-options
-A logaccept -j ACCEPT
-A logdrop -m state --state NEW -j LOG --log-prefix "DROP " --log-tcp-sequence --log-tcp-options --log-ip-options
-A logdrop -j DROP
COMMIT
# Completed on Mon Jul 6 21:33:17 2015
And the trace when it is actually working:
Jul 6 21:30:22 kernel: TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: mangle:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: nat:PREROUTING:rule:1 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: nat:VSERVER:rule:1 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote ip] DST=[public ip] LEN=64 TOS=0x00 PREC=0x00 TTL=53 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: mangle:FORWARD:policy:1 IN=eth0 OUT=br0 SRC=[remote ip] DST=192.168.1.110 LEN=64 TOS=0x00 PREC=0x00 TTL=52 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: filter:FORWARD:rule:5 IN=eth0 OUT=br0 SRC=[remote ip] DST=192.168.1.110 LEN=64 TOS=0x00 PREC=0x00 TTL=52 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: mangle:POSTROUTING:policy:1 IN= OUT=br0 SRC=[remote ip] DST=192.168.1.110 LEN=64 TOS=0x00 PREC=0x00 TTL=52 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: nat:POSTROUTING:policy:3 IN= OUT=br0 SRC=[remote ip] DST=192.168.1.110 LEN=64 TOS=0x00 PREC=0x00 TTL=52 ID=22999 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911719 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C01030306040201010101080AB1AEFA2800000000)
Jul 6 21:30:22 kernel: TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote ip] DST=[public ip] LEN=52 TOS=0x00 PREC=0x00 TTL=53 ID=23000 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911720 ACK=1249233378 WINDOW=4098 RES=0x00 ACK URGP=0 OPT (0101080AB1AEFA29035BF19C)
Jul 6 21:30:23 kernel: TRACE: mangle:PREROUTING:policy:2 IN=eth0 OUT= MAC=f0:79:59:76:27:50:d0:7e:28:75:97:a3:08:00 SRC=[remote ip] DST=[public ip] LEN=52 TOS=0x00 PREC=0x00 TTL=53 ID=23000 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911720 ACK=1249233378 WINDOW=4098 RES=0x00 ACK URGP=0 OPT (0101080AB1AEFA29035BF19C)
Jul 6 21:30:23 kernel: TRACE: mangle:FORWARD:policy:1 IN=eth0 OUT=br0 SRC=[remote ip] DST=192.168.1.110 LEN=52 TOS=0x00 PREC=0x00 TTL=52 ID=23000 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911720 ACK=1249233378 WINDOW=4098 RES=0x00 ACK URGP=0 OPT (0101080AB1AEFA29035BF19C)
Jul 6 21:30:23 kernel: TRACE: filter:FORWARD:rule:1 IN=eth0 OUT=br0 SRC=[remote ip] DST=192.168.1.110 LEN=52 TOS=0x00 PREC=0x00 TTL=52 ID=23000 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911720 ACK=1249233378 WINDOW=4098 RES=0x00 ACK URGP=0 OPT (0101080AB1AEFA29035BF19C)
Jul 6 21:30:23 kernel: TRACE: mangle:POSTROUTING:policy:1 IN= OUT=br0 SRC=[remote ip] DST=192.168.1.110 LEN=52 TOS=0x00 PREC=0x00 TTL=52 ID=23000 DF PROTO=TCP SPT=46858 DPT=5522 SEQ=4293911720 ACK=1249233378 WINDOW=4098 RES=0x00 ACK URGP=0 OPT (0101080AB1AEFA29035BF19C)
What I need help with is figuring out why the package doesn't reach the mangle:FORWARD part of the iptable setup, and how to get it to reach 192.168.1.110.
Thanks to anyone looking at this, it has been bugging me for almost a week.
So, the first problem was that I was blocked by rp_filter. Disabling that for that interface solved that issue. The packages did not, however, leave the machine. For that to work I had to set up a seperate routing table. All and all, it ended up in a script:
#!/bin/sh
WAN_IP=$(ifconfig eth0 | egrep -o 'addr:[0-9.]*' | cut -d ':' -f 2)
PUBLIC_IPS=$(iptables -t nat -L VSERVER | egrep '^DNAT' | egrep -o 'to:[0-9.]*' | cut -d ':' -f 2)
DEFAULT_ROUTE=$(ip route show | egrep -o '^default .* eth0 ')
LAN_ROUTE=$(ip route show | egrep ' br0 ')
echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter
ip route add $DEFAULT_ROUTE table 200
ip route add $LAN_ROUTE table 200
ip rule add fwmark 0xb00b table 200
for IP in $PUBLIC_IPS ; do
iptables -t mangle -I PREROUTING -m conntrack --ctstate DNAT -s "$IP" -i br0 -j CONNMARK --restore-mark
done
iptables -I FORWARD -o eth0 -m state --state NEW -j DROP
iptables -t nat -I PREROUTING -m mark --mark 0 -d "$WAN_IP" -i eth0 -j CONNMARK --set-mark 0xb00b
iptables -t nat -I VSERVER -m mark ! --mark 0xb00b -j VUPNP
iptables -t nat -A VUPNP -j CONNMARK -m mark --mark 0xb00b --set-mark 0
iptables -t mangle -I PREROUTING -m conntrack --ctstate DNAT -d "$WAN_IP" -i eth0 -j CONNMARK --restore-mark
The script searches through the VSERVER rules in the nat table and allows any host in it to be contacted via the VSERVER posts outside of the VPN connection.
The script also separates so that UPNP connections are open only to the VPN and VSERVER connections are open only to the public IP.
I hope this helps somebody else too.