Search code examples
dockersslproxycloudflaremitmproxy

Mitmproxy only proxying first page load

Background

I have created the following setup with WireGuard, mitmproxy and redsocks in order to be able to use a commercial VPN connection for all outgoing traffic while still being able to intercept/record/modify all my traffic, no matter where I am or which device I am using.

My desired topology looks like this: enter image description here

I have used the following posts for guidance:

Code

docker-compose.yml
version: "3.9"

services:
  wg-in:
    image: "linuxserver/wireguard:1.0.20210914"
    container_name: wireguard-in
    cap_add:
      - NET_ADMIN
    restart: always
    networks:
      wireguard_in_backend:
        ipv4_address: "172.20.1.2"
    environment:
      - PUID=1000
      - GUID=1000
      - TZ=Etc/UTC
      - PEERS=laptop,phone,general
      - INTERNAL_SUBNET=10.10.10.0
    ports:
      - 51820:51820/udp
    volumes:
      - ./config-in:/config
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1

  wg-out:
    image: "linuxserver/wireguard:1.0.20210914"
    container_name: wireguard-out
    cap_add:
      - NET_ADMIN
    restart: always
    networks:
      wireguard_out_backend:
        ipv4_address: "172.21.1.2"
    environment:
      - PUID=1000
      - GUID=1000
      - TZ=Etc/UTC
    volumes:
      - ./config-out:/config
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1

  mitm:
    image: derbob/mitmproxy:latest
    container_name: wireguard-mitm
    command: ["/bin/bash", "/app/run.sh"]
    cap_add:
      - NET_ADMIN
    restart: always
    networks:
      wireguard_out_backend:
        ipv4_address: "172.21.1.3"
    volumes:
      - ./mitmproxy/.mitmproxy:/.mitmproxy
      - ./dev/mitm/run.sh/:/app/run.sh

  redsocks:
    image: derbob/redsocks:latest
    container_name: wireguard-redsocks
    cap_add:
      - NET_ADMIN
    restart: always
    networks:
      wireguard_in_backend:
        ipv4_address: "172.20.1.4"
      wireguard_out_backend:
        ipv4_address: "172.21.1.4"
    volumes:
      - ./dev/redsocks/redsocks.conf:/etc/redsocks.conf
      - ./dev/redsocks/run.sh:/app/run.sh

networks:
  wireguard_in_backend:
    name: "wireguard_in_backend"
    ipam:
      driver: default
      config:
        - subnet: "172.20.0.0/16"
  wireguard_out_backend:
    name: "wireguard_out_backend"
    ipam:
      driver: default
      config:
        - subnet: "172.21.0.0/16"

Wireguard

./config-in/templates/server.conf
[Interface]
Address = ${INTERFACE}.1
ListenPort = 51820
PrivateKey = $(cat /config/server/privatekey-server)
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE

PostUp = wg set wg0 fwmark 51820
PostUp = ip -4 route add 0.0.0.0/0 via 172.20.1.4 table 51820
PostUp = ip -4 rule add not fwmark 51820 table 51820
PostUp = ip -4 rule add table main suppress_prefixlength 0

PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth+ -j MASQUERADE

redsocks

Dockerfile derbob/redsocks:latest
FROM debian:latest
WORKDIR /app
ADD . /app
RUN apt-get update
RUN apt-get upgrade -qy
RUN apt-get install iptables redsocks curl wget iproute2 -qy
COPY redsocks.conf /etc/redsocks.conf
ENTRYPOINT /bin/bash run.sh
./dev/redsocks/redsocks.conf
base {
 log_debug = on;
 log_info = on;
 log = stderr;
 daemon = off;
 user = redsocks;
 group = redsocks;
 redirector = iptables;
}
redsocks {
 local_ip = 0.0.0.0;
 local_port = 12345;
 ip = 172.21.1.3;
 port = 8080;
 type = http-connect;
}
./dev/redsocks/run.sh
#!/bin/bash
echo "Set iptables rules"
# to route outside connections, we need to tell the container to properly route them
iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE

# Create REDSOCKS chain
iptables -t nat -N REDSOCKS

# Exclude local and reserved addresses from Redsocks chain
iptables -t nat -A REDSOCKS -d 0.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 10.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 127.0.0.0/8 -j RETURN
iptables -t nat -A REDSOCKS -d 169.254.0.0/16 -j RETURN
iptables -t nat -A REDSOCKS -d 172.16.0.0/12 -j RETURN
iptables -t nat -A REDSOCKS -d 192.168.0.0/16 -j RETURN
iptables -t nat -A REDSOCKS -d 224.0.0.0/4 -j RETURN
iptables -t nat -A REDSOCKS -d 240.0.0.0/4 -j RETURN

# Redirect all packets from REDSOCKS chain to the local port
iptables -t nat -A REDSOCKS -p tcp -j REDIRECT --to-ports 12345

# Redirect all HTTP and HTTPS outgoing packets through Redsocks
iptables -t nat -A OUTPUT -p tcp --dport 443 -j REDSOCKS
iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDSOCKS

# Redirect incoming packets to the REDSOCKS chain
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDSOCKS
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDSOCKS
iptables -t nat -A PREROUTING -p tcp --dport 1080 -j REDSOCKS

# Route all traffic destined outside docker bridges to outgoing Wireguard client
ip route del default
ip route add default via 172.21.1.2

echo "Starting redsocks ..."
redsocks -c /etc/redsocks.conf

mitmproxy

Dockerfile derbob/mitmproxy:latest
FROM mitmproxy/mitmproxy:9.0.1   
                                 
RUN apt-get update               
RUN apt-get upgrade -qy          
RUN apt-get install iproute2 -qy
./dev/mitm/run.sh
#!/bin/bash
ip route del default
ip route add default via 172.21.1.2

mitmdump --set confdir=/.mitmproxy

Issue

This setup works like I intended it for the most part. HTTP traffic passes through the proxy just fine, as does most HTTPS traffic. I seem to have an issue though when the remote host of the SSL connection is using Cloudflare. In that case, only the initial HTTPS connection is being routed through mitmproxy. Subsequent requests to the same host within the same browser window are not proxied anymore.

I have installed the mitmproxy CA cert the official way by going through mitm.it. Here are the steps I then take to observe this behavior:

  1. Open a new Chromium Incognito window
  2. Navigate to https://icanhazip.com
  3. Check the certificate info

enter image description here

  1. Hit refresh
  2. Check the certificate info again

enter image description here

On the second request, my broswer seems to be talking directly to the Cloudflare server, bypassing mitmproxy alltogether. This behavior is consistent for all websites that use Cloudflare.

I have done some more testing around this and come up with the following results:

  • when I use an SSL site that does not utilize Cloudflare (e.g. https://api.ipify.com), all requests pass through mitmproxy and mitmproxy is shown always shown as CA

  • when I use a device that has not added the mitmproxy CA cert, all https requests to https://icanhazip.com are proxied by mitmproxy

  • when I use another MITM proxy on my local machine (i.e. OWASP ZAP) before the traffic hits the first WireGuard tunnel, all https requests to https://icanhazip.com are again proxied by mitmproxy

  • when I close the Incognito window and open a new one, the first requet to https://icanhazip.com is again proxied while the following ones aren't

  • when I remove these two lines from ./dev/redsocks/run.sh

    ip route del default
ip route add default via 172.21.1.2

    the first request to https://icanhazip.com in a fresh incognito window shows the IP address of my VPN service, while all subsequent ones show the IP address of my server, confirming that the traffic goes directly from the redsocks container to the remote host and thus bypassing mitmproxy

Theory

The results from my tests lead me to believe that somehow SSL connections to Cloudflare-enabled sites open up a separate channel of communication which is no longer routed from the redsocks to the mitmproxy container. This theory is supported by results of the different tests I made:

  • it does not happen for SSL connections to sites that don't use Cloudflare
  • the fact that the first request in a fresh browser session is always proxied
  • the IP change in the last test
  • although I don't know how or why, using another MITM proxy locally and/or not trusting the mitmproxy CA cert seem to prevent this separate communications channel from being opened

The only way I see this can happen is if the communication no longer happens on port 443, in which case it would no longer be forwarded from redsocks to mitmproxy. I haven't found a way to verify this theory yet. I messed around with Wireshark but didn't know where to look. redsocks also seems to not be producing any meaningful logging output. This is all I get:

1690996775.715401 info redsocks.c:1243 redsocks_accept_client(...) [172.20.1.2:5434->104.18.
114.97:443]: accepted
1690996775.734750 debug redsocks.c:341 redsocks_start_relay(...) [172.20.1.2:5434->104.18.11
4.97:443]: data relaying started
1690996780.945461 info redsocks.c:671 redsocks_drop_client(...) [172.20.1.2:5434->104.18.114
.97:443]: connection closed

The first two messages happen upon the first request to https://icanhazip.com and the last one when I close my browser window. There is no interaction logged inbetween, but this behavior is consistent for all remote hosts.

Solution

  • Modern browsers support HTTP/3 which is using UDP with default port 443. Cloudflare supports HTTP/3 for the sites they serve. Since you don't redirect UDP port 443 and since redsocks and mitmproxy would be unable to deal with this traffic anyway it bypasses your interception.

    To fix this you need to simply block access to UDP port 443. The browser will then fall back to HTTP/2 or HTTP/1 which you can catch.