Why does my proxied docker network request work locally but not in production?

Question

I'm working on a project to build a front end for a private/secure docker registry. The way I'm doing this is to use docker-compose to create a network between the front end and the registry. My idea is to use express to serve my site and forward requests from the client to the registry via the docker network.

Locally, everything works perfectly....

However, in production the client doesn't get a response back from the registry. I can login to the registry and access it's api via postman (for ex the catalog) at https://myregistry.net:5000/v2/_catalog. But... the client just errors out.

when I go into the express server container and try to curl the endpoint I created to proxy requests, I get this

curl -vvv http://localhost:3000/api/images
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3000 (#0)
> GET /api/images HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.61.1
> Accept: */*
>
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server

and the error that's returned includes a _currentUrl of https://username:password@registry:5000/v2/_catalog

my docker-compose file looks like this...

version: '3'
services:
  registry:
    image: registry:2
    container_name: registry
    ports:
      # forward requests to registry.ucdev.net:5000 to 127.0.0.1:443 on the container
      - "5000:443"
    environment:
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
      REGISTRY_HTTP_ADDR: 0.0.0.0:443
      REGISTRY_HTTP_TLS_CERTIFICATE: /certs/fullchain.pem
      REGISTRY_HTTP_TLS_KEY: /certs/privkey.pem
    volumes:
      - /etc/letsencrypt/live/registry.ucdev.net/fullchain.pem:/certs/fullchain.pem
      - /etc/letsencrypt/live/registry.ucdev.net/privkey.pem:/certs/privkey.pem
      - ./auth:/auth
    restart: always
  server:
    image: uc/express
    container_name: registry-server
    ports:
      - "3000:3000"
    volumes:
      - ./:/project
    environment:
      NODE_ENV: production
    restart: always
    entrypoint: ["npm", "run", "production"]

an example of my front end request looks like this...

axios.get('http://localhost:3000/api/images')
      .then((response) => {
        const { data: { registry, repositories } } = response;
        this.setState((state, props) => {
          return { registry, repositories }
        })
      })
      .catch((err) => {
        console.log(`Axios error -> ${err}`)
        console.error(err)
      })

and that request is sent to the express server and then to the registry like this...

app.get('/api/images', async (req, res) => {
  // scheme is either http or https depending on NODE_ENV
  // registry is the name of the container on the docker network
  await axios.get(`${scheme}://registry:5000/v2/_catalog`)
  .then((response) => {
    const { data } = response;
    data.registry = registry;
    res.json(data);
  })
  .catch((err) => {
    console.log('Axios error -> images ', err);
    return err;
  })
})

any help you could offer would be great! thanks!

Answer 1

In this particular case it was an issue related to the firewall the server was behind. requests coming from the docker containers were being blocked. to solve this problem we had to explicitly set the network_mode to bridge. this allowed requests to from within the containers to behave correctly. the final docker-compose file looks like this

version: '3'
services:
  registry:
    image: registry:2
    container_name: registry
    # setting network_mode here and on the server helps the express api calls work correctly on the myregistry.net server.
    # otherwise, the calls fail with 'network unreachable' due to the firewall.
    network_mode: bridge
    ports:
      # forward requests to myregistry.net:5000 to 127.0.0.1:443 on the container
      - "5000:443"
    environment:
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
      REGISTRY_HTTP_ADDR: 0.0.0.0:443
      REGISTRY_HTTP_TLS_CERTIFICATE: /certs/fullchain.pem
      REGISTRY_HTTP_TLS_KEY: /certs/privkey.pem
    volumes:
      - /etc/letsencrypt/live/myregistry.net/fullchain.pem:/certs/fullchain.pem
      - /etc/letsencrypt/live/myregistry.net/privkey.pem:/certs/privkey.pem
      - ./auth:/auth
    restart: always
  server:
    image: uc/express
    container_name: registry-server
    network_mode: bridge
    ports:
      - "3000:3000"
    volumes:
      - ./:/project
    environment:
      NODE_ENV: production
    restart: always
    entrypoint: ["npm", "run", "production"]