Search code examples
linuxdockerchroot

Using ip link To Connect Multiple Chroot Jails

I've done some research online, and I'm not really finding any info about how to establish a communication link between two or more chroot jails. The "standard" seems to be just using a single chroot jail for sandboxing.

To put things into context, I'm trying to perform the basic functions of docker containers, but with chroot jails. So, in the same way that two docker containers can ping each other via IP and/or be connected on the same user-defined docker network, can this exist for two chroot jails?

I understand that containers and chroot jails are different things, and I'm familiar with both. I just need to know if there is a way to link two chroot jails in a similar way to linking two containers, or if this is nonexistent and I'm just wasting my time

Solution

  • There's nothing magic about Docker: it's just using the facilities provided by the Linux kernel to enable various sorts of isolation. You can take advantage of the same features.

    A Docker "network" is nothing more than a bridge device on your host. You can create those trivially using either brctl or the ip link command, as in:

    ip link add mynetwork type bridge

    You'll want to activate the interface and assign an ip address:

    ip addr add 192.168.23.1/24 dev mynetwork
ip link set mynetwork up

    A Docker container has a separate network environment from the host. This is called a network namespace, and you can manipulate them by hand with the ip netns command.

    You can create a network namespace with the ip netns add command. For example, here we create two namespaces named chroot1 and chroot2:

    ip netns add chroot1
ip netns add chroot2

    Next, you'll create two pairs of veth network interfaces. One end of each pair will be attached to one of the above network namespaces, and the other will be attached to the mynetwork bridge:

    # create a veth-pair named chroot1-inside and chroot1-outside
ip link add chroot1-inside type veth peer name chroot1-outside
ip link set master mynetwork dev chroot1-outside
ip link set chroot1-outside up
ip link set netns chroot1 chroot1-inside

# do the same for chroot2
ip link add chroot2-inside type veth peer name chroot2-outside
ip link set netns chroot2 chroot2-inside
ip link set chroot2-outside up
ip link set master mynetwork dev chroot2-outside

    And now configure the interfaces inside of the network namespaces. We can do this using the -n option to the ip command, which causes the command to run inside of the specified network namespace:

    ip -n chroot1 addr add 192.168.23.11/24 dev chroot1-inside
ip -n chroot1 link set chroot1-inside up

ip -n chroot2 addr add 192.168.23.12/24 dev chroot2-inside
ip -n chroot2 link set chroot2-inside up

    Note that in the above, ip -n <namespace> is just a shortcut for:

    ip netns exec <namespace> ip ...

    So that:

    ip -n chroot1 link set chroot1-inside up

    Is equivalent to:

    ip netns exec chroot1 ip link set chroot1-inside up

    (Apparently older versions of the iproute package don't include the -n option.)

    And finally, you can start your chroot environments inside of these two namespaces. Assuming that your chroot filesystem is mounted on /mnt, in one terminal, run:

    ip netns exec chroot1 chroot /mnt bash

    And in another terminal:

    ip netns exec chroot2 chroot /mnt bash

    You will find that these two chroot environments can ping each other. For example, inside the chroot1 environment:

    # ip addr show
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
36: chroot1-inside@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 12:1c:9c:39:22:fa brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.23.11/24 scope global chroot1-inside
       valid_lft forever preferred_lft forever
    inet6 fe80::101c:9cff:fe39:22fa/64 scope link 
       valid_lft forever preferred_lft forever

# ping -c1 192.168.23.12
PING 192.168.23.12 (192.168.23.12) 56(84) bytes of data.                
From 192.168.23.1 icmp_seq=1 Destination Host Prohibited                

--- 192.168.23.12 ping statistics ---                                   
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

    And they can ping the host:

    # ping -c1 192.168.23.1                                    
PING 192.168.23.1 (192.168.23.1) 56(84) bytes of data.     
64 bytes from 192.168.23.1: icmp_seq=1 ttl=64 time=0.115 ms

--- 192.168.23.1 ping statistics ---                       
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.115/0.115/0.115/0.000 ms

    Of course, for this chroot environment to be useful, there's a bunch of stuff missing, such as the virtual filesystems on /proc, /sys, and /dev. Setting all of this up and tearing it back down cleanly is why people use tools like Docker, or systemd-nspawn, because there's a lot to manage by hand.