Search code examples
dockernetwork-programmingsynology

Docker in bridge network mode on ARM-based Synology NAS?

I have an ARM64-based Synology NAS device and have been trying to set up Docker on it using the instructions found here:

Can I install Docker on arm8 based Synology Nas

However, the fact that I can't use the default bridge network mode but instead have to use host mode (network_mode=host) is preventing me from doing some things that I'd like to do. In the aforementioned thread user P Leo writes:

Please note, you need to set storage drive vfs, iptables off, bridge off due to a Linux kernel problem. And you need to run docker container with --network=host mode. It is not usual, but it is necessary due to Synology NAS kernel limitations.

I was wondering if anyone could shed more light on this apparent limitation? Based on other Synology- and Docker-related discussions online, it seems that it doesn't affect some users. Is the issue perhaps limited to ARM-based devices or specific Linux kernel versions (my device has 4.4.180+)? And most importantly, is there really no way around it?

Thanks in advance for any help!

Solution

  • I've done this recently on a DS218 that has an arm64 (RTD1296) CPU. It's a bit involved but doable.

    Start Docker in bridge Mode

    First thing, check if your NAS has the necessary kernel modules in /usr/lib/modules. Mine did, so load them in this order:

      sudo insmod /usr/lib/modules/veth.ko
  sudo insmod /usr/lib/modules/stp.ko
  sudo insmod /usr/lib/modules/tun.ko
# sudo insmod /usr/lib/modules/nf_defrag_ipv4.ko # Already loaded for me
# sudo insmod /usr/lib/modules/nf_conntrack.ko
# sudo insmod /usr/lib/modules/x_tables.ko
  sudo insmod /usr/lib/modules/xt_TCPMSS.ko
# sudo insmod /usr/lib/modules/xt_recent.ko
  sudo insmod /usr/lib/modules/xt_NFQUEUE.ko
  sudo insmod /usr/lib/modules/xt_mark.ko
# sudo insmod /usr/lib/modules/xt_mac.ko
# sudo insmod /usr/lib/modules/xt_limit.ko
# sudo insmod /usr/lib/modules/xt_iprange.ko
# sudo insmod /usr/lib/modules/xt_geoip.ko
  sudo insmod /usr/lib/modules/xt_addrtype.ko
  sudo insmod /usr/lib/modules/xt_conntrack.ko
# sudo insmod /usr/lib/modules/xt_LOG.ko
  sudo insmod /usr/lib/modules/bridge.ko
  sudo insmod /usr/lib/modules/br_netfilter.ko
# sudo insmod /usr/lib/modules/xt_state.ko
# sudo insmod /usr/lib/modules/xt_tcpudp.ko
# sudo insmod /usr/lib/modules/xt_multiport.ko
  sudo insmod /usr/lib/modules/nf_nat.ko
# sudo insmod /usr/lib/modules/nf_conntrack_ipv4.ko
# sudo insmod /usr/lib/modules/ip_tables.ko
  sudo insmod /usr/lib/modules/nf_nat_redirect.ko
  sudo insmod /usr/lib/modules/xt_REDIRECT.ko
  sudo insmod /usr/lib/modules/xt_nat.ko
  sudo insmod /usr/lib/modules/nf_nat_ipv4.ko
  sudo insmod /usr/lib/modules/nf_nat_masquerade_ipv4.ko
  sudo insmod /usr/lib/modules/ipt_MASQUERADE.ko
# sudo insmod /usr/lib/modules/iptable_filter.ko
  sudo insmod /usr/lib/modules/iptable_nat.ko

    (If you see a File Exists error, ignore it; it just means the module has already been loaded.)

    Then check if iptables can match by addrtype or conntrack since dockerd needs both.

    iptables -m addrtype -h
iptables -m conntrack -h

    If you see an error such as No such file or directory, you have work to do. You can either install a version of iptables that's not crippled from Entware (opkg install iptables), or you can compile the missing libs yourself. I did the latter on my Mac:

    % docker run --name=ubuntu -it --rm --platform linux/arm64 --entrypoint bash ubuntu:latest
% apt-get update && apt-get install build-essential bc bison flex vim wget xz-utils
% mkdir /syno && cd /syno
### Note: version 1.6.0 was shipped with my NAS
% wget https://www.netfilter.org/projects/iptables/files/iptables-1.6.0.tar.bz2
### Note: also needed these dependent packages
% wget https://www.netfilter.org/projects/libnftnl/files/libnftnl-1.0.5.tar.bz2
% wget https://www.netfilter.org/projects/libmnl/files/libmnl-1.0.3.tar.bz2
% tar -xjf *.tar.bz2
% cd iptables-1.6.0
% cp ../libnftnl-1.0.5/include/libnftnl include/
% cp ../libmnl-1.0.3/include/libmnl include/
% ./configure --host=arm-linux-gnueabi --target=arm-linux-gnueabi --prefix=/syno
% make
% find /syno -name libxt_*so

    The compile failed for me, but it produced the 2 libs libxt_addrtype.so and libxt_conntrack.so I needed. Copy them to /usr/lib/iptables/ on your NAS, the 2 commands above should run without errors.

    And now dockerd should be able to start successfully with bridge network:

    sudo cat <<EOF >/etc/docker/daemon.json
{
    "storage-driver": "vfs",
    "bip": "172.16.0.1/16",
    "default-gateway": "172.16.0.254"
}
EOF
sudo /bin/dockerd &

    Turn on IP Forwarding and Configure Firewall

    To be able to communicate with your containers from your LAN, you also need to do these to allow IP forwarding:

    # Turn on IP forwarding
sudo sysctl -w net.ipv4.ip_forward=1
sudo sysctl -w net.ipv6.conf.all.forwarding=1
sudo iptables -P FORWARD ACCEPT
# Allow containers to access Docker socket
sudo chmod 666 /var/run/docker.sock

    And finally, open up your firewall to allow access to the ports. Assuming you have -p 8080:80 for a container, you'll need to open port 8080 for your LAN the NAS is in (eg. 192.168.0.0/24).

    Configure a Proxy

    With the above though, what I found was, if I ran a container with published ports, I still could only reach the server via localhost, but not via the LAN IP of my NAS.

    sudo docker run \
    --name=nginx-test \
    --rm \
    --network=bridge \
    -e PUID=`id -u $USER` \
    -e PGID=`id -g $USER` \
    -p 8080:80 \
    -v /volume1/docker/nginx:/usr/share/nginx/html:ro \
    -v /dev:/dev \
    nginx:latest

curl localhost:8080    # This succeeded
curl 192.168.0.10:8080 # This timed out

    Docker was supposed to take care of this for me, but somehow it didn't.

    So I needed a proxy to forward packets (TCP in addition to HTTP/S, in my use case) between 192.168.0.10:8080 and localhost:8080. I opted to use nginx because I couldn't get either iptables or docker-proxy to work, but I had to change my LAN access to a new port, ie. 192.168.0.10:8081.

    opkg install nginx # Needed v1.9, the one shipped was too old
cat <<EOF >/opt/etc/nginx/nginx.conf
user nobody users;
stream {
    upstream server_8080 {
        server localhost:8080;
    }
    server {
        listen 8081;
        proxy_pass server_8080;
    }
}
sudo /opt/etc/init.d/S80nginx start

    Finally after also changing the open port in firewall to 8081, curl 192.168.0.10:8081 worked.

    PS. If someone could tell me why iptables or docker-proxy didn't forward ports between localhost and my LAN IP, I'll appreciate it.