I need a containerto have ports on the host available so it can always be connected to on the same hostname, but also a secondary network connection that gives the container its own LAN IP (via macvlan).
Is that technically possible with both of those on 1 network interface?
The main interface for the machine is fbond0, the macvlan network was created on top of that interface:
docker network create -d macvlan --subnet 10.123.56.0/22 --ip-range=10.123.59.64/27 -o macvlan_mode=bridge -o parent=bond25g macvlan214
When the container starts normally with port mappings, it works as expected:
PORTS
0.0.0.0:49215->22/tcp, 0.0.0.0:49216->5000/tcp
As soon as it's connected to the 2nd network:
PORTS
22/tcp, 5000/tcp
After disconnect, we get new mappings:
PORTS
0.0.0.0:49217->22/tcp, 0.0.0.0:49218->5000/tcp
We already use this setup on a different server, but since that one is outside of the relevant VLAN, a (VLAN'ed) network interface was added, so the exposed ports are not on the same interface as the macvlan functionality far as the OS is concerned.
I'm assuming we'll probably need a similar solution here? Add a 2nd virtual network interface so we can separate things?
For future me and anyone else experiencing this problem, apparently it has to do with how libnetwork chooses a default gateway. Normally this would be based on the interface priority, but given that you cannot change the priority from the configuration, they decided on "choosing the first network in alphabetically order".
Basically, because my macvlan came earlier alphabetically sorted, it was chosen as the default network. Because a macvlan is a lot like host networking, it does not support port mapping. Hence the ports were removed.