Search code examples
linuxnetworkingiproute

Linux ip routing with multiple uplinks SINGLE interface


I'm trying to setup a Proxmox machine that is running 3 VMs. it has 3 public ips but these ips are on a single interface (eth0).

the 3 vms are on a bridge (vmbr0) with an address of 172.16.0.1/24.

I have enable ip masquerading and forwarding. but I cannot figure out how to make each of the 3 vms (172.16.0.2, 172.16.0.3, 172.16.0.4) route out through a specific one of the public ips.

I have tried the standard iproute with 3 tables setting the gateways and rules but no matter what rule I set the vms still route out through the primary ip.

trouble is the 3 public ips are on complete separate networks so they each have a different gateway. I know how to use iproute to do this if each public ip was on a separate physical interface but this machine has all 3 on a single interface and iproute doesn't seem to like that because if I do ip route add default via 23.92.26.1 dev eth0:2 table node2 and then later list everything it shows via eth0. so apparently iproute doesn't like pseudo interfaces. I don't know a lot about iptables and I'm sure there's a way to do this with pure iptables but haven't found anything. all my google searches come up with iproute tables which like I said don't seem to work with a single interface.

Thank you in advance


Solution

  • considering ProxMox is running Debian try adding something like the following to your /etc/network/interfaces for each of the extra links

    post-up route add -net <network identifier> netmask <netmask> gw <links gateway>
    pre-down route del -net <network identifier> netmask <netmask> gw <links gateway>
    

    and then with iptables if you want 172.16.0.2 to go through the second ip do like the following: (this is called Source NAT or SNAT) the --to-source specifies what ip you want the outgoing connections remapped to.

    iptables -t nat -A POSTROUTING -s 172.16.0.2/24 -j SNAT --to-source <ip address you want it to go out of>
    

    if you want all incoming connections on the same ip to go to 172.16.0.2 then you would also add the following (this is called Destination NAT or DNAT)

    iptables -t nat -A PREROUTING -d <ip/mask bit> -j DNAT --to-destination 172.16.0.2