Caddy as reverse proxy in docker refuses to connect to other containers

Question

I wanted to try out Caddy in a docker environment but it does not seem to be able to connect to other containers. I created a network "caddy" and want to run a portainer alongside it. If I go into the volume of caddy, I can see, that there are certs generated, so that seems to work. Also portainer is running and accessible via the Server IP (http://65.21.139.246:1000/). But when I access via the url: https://smallhetzi.fading-flame.com/ I get a 502 and in the log of caddy I can see this message:

{
    "level": "error",
    "ts": 1629873106.715402,
    "logger": "http.log.error",
    "msg": "dial tcp 172.20.0.2:1000: connect: connection refused",
    "request": {
        "remote_addr": "89.247.255.231:15146",
        "proto": "HTTP/2.0",
        "method": "GET",
        "host": "smallhetzi.fading-flame.com",
        "uri": "/",
        "headers": {
            "Accept-Encoding": [
                "gzip, deflate, br"
            ],
            "Accept-Language": [
                "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
            ],
            "Cache-Control": [
                "max-age=0"
            ],
            "User-Agent": [
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
            ],
            "Sec-Fetch-Site": [
                "none"
            ],
            "Accept": [
                "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
            ],
            "Sec-Fetch-Mode": [
                "navigate"
            ],
            "Sec-Fetch-User": [
                "?1"
            ],
            "Sec-Fetch-Dest": [
                "document"
            ],
            "Sec-Ch-Ua": [
                "\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\""
            ],
            "Sec-Ch-Ua-Mobile": [
                "?0"
            ],
            "Upgrade-Insecure-Requests": [
                "1"
            ]
        },
        "tls": {
            "resumed": false,
            "version": 772,
            "cipher_suite": 4865,
            "proto": "h2",
            "proto_mutual": true,
            "server_name": "smallhetzi.fading-flame.com"
        }
    },
    "duration": 0.000580828,
    "status": 502,
    "err_id": "pq78d9hen",
    "err_trace": "reverseproxy.statusError (reverseproxy.go:857)"
}

But two compose files:

Caddy:

version: '3.9'

services:
  caddy:
    image: caddy:2-alpine
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - certs-volume:/data
      - caddy_config:/config

volumes:
  certs-volume:
  caddy_config:

networks:
  default:
    external:
      name: caddy

Caddyfile:

{
    email [email protected]
    # acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}

smallhetzi.fading-flame.com {
    reverse_proxy portainer:1000
}

and my portainer file:

version: '3.9'

services:
  portainer:
    image: portainer/portainer-ce
    container_name: portainer
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data portainer/portainer
    entrypoint: /portainer -p :80
    ports:
      - "1000:80"

volumes:
  portainer_data:

networks:
  default:
    external:
      name: caddy

What I think happens is, that those two containers are somehow not in the same network, but I dont get why.

What works as a workaround right now is, when i make this change to my Caddyfile:

smallhetzi.fading-flame.com {
    reverse_proxy 65.21.139.246:1000
}

Then I get a valid certificate and the portainer ui. But i would rather not spread the IPs over my Caddyfile. Do I have to configure something else for caddy to run in docker?

Answer 1

I just got help from the forum and it turns out, that caddy redirects to the port INSIDE the container, not the public one. In my case, portainer runs on 80 internally, so changing the Caddyfile to this:

smallhetzi.fading-flame.com {
    reverse_proxy portainer:80
}

or this

smallhetzi.fading-flame.com {
    reverse_proxy http://portainer
}

does the job. This also means, that I could get rid of exposing portainer directly over the port 1000. Now I can only access it via the proxy.

Hope someone gets some help from that :)