Search code examples
dockerroutesreverse-proxydocker-swarmtraefik

Service in Traefik path does not redirect to service path for additional requests

I have a VM with a subdomain - e.g. mysubdomain.notmydomain.com, as visible in the logs below - that runs a single node Docker Swarm (for simplicity) and uses Traefik as a reverse proxy for various services. I managed to simplify my issues with different services to the one I am sharing here now.

Expected behaviour: I got to mysubdomain.notmydomain.com/test and I see a response with text and a rendered logo.

Actual behaviour:: I see a broken link (see photo)

Versions

Ubuntu 16.04.1 LTS xenial

Docker version 18.03.0-ce, build 0520e24

Traefik 1.5.4

Truncated docker-compose file

version: '3.4'

services:
  test:
    image: tutum/hello-world
    networks:
      - traefik-network
    deploy:
      labels:
        traefik.enable: "true"
        traefik.backend: "test"
        traefik.docker.network: "cat_traefik-network"
        traefik.frontend.rule: "Host:mysubdomain.notmydomain.com;PathPrefix:/test"
        traefik.port: 80
        traefik.frontend.passHostHeader: "true"
        traefik.domain: "mysubdomain.notmydomain.com"
  traefik:
    image: traefik
    ports: ["80:80", "443:443", "8080:8080"]
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - ./traefik/traefik.toml:/traefik.toml
    - ./traefik/acme.json:/acme.json
    networks:
    - traefik-network
    - mosquitto-network
    command:
      --api \
      --docker \
      --docker.swarmmode \
      --docker.domain=mysubdomain.notmydomain.com \
      --docker.watch \
      --logLevel=DEBUG
    deploy:
      placement:
        constraints: [node.role==manager]
      labels:
       traefik.enable: "false"

Screenshot showing unexpected behaviour

Proof of error

Logs from Traefik

cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{"Method":"GET","URL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/test","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Cache-Control":["max-age=0"],"Connection":["keep-alive"],"Cookie":["redirect_to=%252F"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"]},"ContentLength":0,"TransferEncoding":null,"Host":"mysubdomain.notmydomain.com","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.255.0.2:54617","RequestURI":"/test","TLS":null}"
cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{"Method":"GET","URL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/test","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Cache-Control":["max-age=0"],"Connection":["keep-alive"],"Cookie":["redirect_to=%252F"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"]},"ContentLength":0,"TransferEncoding":null,"Host":"mysubdomain.notmydomain.com","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.255.0.2:54617","RequestURI":"/test","TLS":null}" ForwardURL=http://10.0.2.7:80
cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/forward: begin ServeHttp on request" Request="{"Method":"GET","URL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.2.7:80","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Cache-Control":["max-age=0"],"Connection":["keep-alive"],"Cookie":["redirect_to=%252F"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"]},"ContentLength":0,"TransferEncoding":null,"Host":"mysubdomain.notmydomain.com","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.255.0.2:54617","RequestURI":"/test","TLS":null}"
cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/forward/http: begin ServeHttp on request" Request="{"Method":"GET","URL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.2.7:80","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Cache-Control":["max-age=0"],"Connection":["keep-alive"],"Cookie":["redirect_to=%252F"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"]},"ContentLength":0,"TransferEncoding":null,"Host":"mysubdomain.notmydomain.com","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.255.0.2:54617","RequestURI":"/test","TLS":null}"
cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/forward/http: Round trip: http://10.0.2.7:80, code: 200, Length: 478, duration: 8.138087ms"
cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/forward/http: completed ServeHttp on request" Request="{"Method":"GET","URL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.2.7:80","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Cache-Control":["max-age=0"],"Connection":["keep-alive"],"Cookie":["redirect_to=%252F"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"]},"ContentLength":0,"TransferEncoding":null,"Host":"mysubdomain.notmydomain.com","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.255.0.2:54617","RequestURI":"/test","TLS":null}"
cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/forward: completed ServeHttp on request" Request="{"Method":"GET","URL":{"Scheme":"http","Opaque":"","User":null,"Host":"10.0.2.7:80","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Cache-Control":["max-age=0"],"Connection":["keep-alive"],"Cookie":["redirect_to=%252F"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"]},"ContentLength":0,"TransferEncoding":null,"Host":"mysubdomain.notmydomain.com","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.255.0.2:54617","RequestURI":"/test","TLS":null}"
cat_traefik.1.ifbq38ugx76r@cat    | time="2018-04-03T20:42:39Z" level=debug msg="vulcand/oxy/roundrobin/rr: competed ServeHttp on request" Request="{"Method":"GET","URL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/test","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Cache-Control":["max-age=0"],"Connection":["keep-alive"],"Cookie":["redirect_to=%252F"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"]},"ContentLength":0,"TransferEncoding":null,"Host":"mysubdomain.notmydomain.com","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.255.0.2:54617","RequestURI":"/test","TLS":null}"

The logs show that the first few request contain Path:/test whereas the last contain Path:/. I assume this is what is going wrong.

Question: How or what do I need to reconfigure to have the secondary request on the tutum/hello-world container, containing the logo.png, load properly?

The tutum/hello-world image is chosing for testing and as a clear example; the actual stack contains a mosquitto, grafana, influxdb, and some custom services that all show the same root behaviour. (those that have frontend.rule's)`

Any suggestions are appreciated; I can provide more information when needed.

Solution

  • From Github Issue #3124 by Julien Salleyron:

    Thanks for your interest in Træfik!

    It's not an issue. Træfik does not rewrite the response body. We think that the process of rewriting the response body in a reverse proxy is too heavy/prone to errors.

    We encourage you to either:

    • use host based rules only (seems impossible for your use case)
    • use specific Traefik path rewrite headers in you app (X-Forwarded-Prefix, X-Replaced-Path)
    • let your app listen on a specific path