Allow outbound container networking through vpnkit

Question

I have a linuxkit built VM here with a custom container service that I am trying to run.

services:
  ...

  - name: net-manager
    image: aemengo/net-manager:6bcc223a83e8a303a004bc6f6e383a54a3d19c55-amd64
    net: host
    capabilities:
      - all
    binds:
      - /usr/bin/vpnkit-expose-port:/usr/bin/vpnkit-expose-port # userland proxy
      - /usr/bin/vpnkit-iptables-wrapper:/usr/bin/iptables # iptables wrapper
      - /var/vpnkit:/port # vpnkit control 9p mount
      - /var/run:/var/run
    command:
      - sleep
      - 1d

With a base image of Alpine, the point of the net-manager service is to allow public internet connectivity to virtual ethernet adapters that I am spinning up on the host: net namespace. My current attempt is the following (inside the container):

$ sysctl net.ipv4.conf.all.forwarding=1
$ /usr/bin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Just like you would do with a VM that wasn't utilizing vpnkit, but there doesn't seem to be any noticeable effect from doing this. For example, nc -v google.com is still failing. What am I missing? vpnkit is mounted and forwarded as the example here instructs to do:

https://github.com/linuxkit/linuxkit/blob/master/examples/docker-for-mac.yml

Answer 1

It turns out that the problem was this line here:

binds:
...

/usr/bin/vpnkit-iptables-wrapper:/usr/bin/iptables

By overriding what the iptables executable was to the one provided by docker, things were misbehaving even though the commands reported no issue. It must be used for something swarm specific, as was mentioned in their docs.

The fix was to remove that binding and run the iptables that was provided in the container