Why can't I use mapped ports to connect two containers in docker compose?

Question

I am new to docker and docker compose and I'm just trying to understand it better.

I was having the following problem when trying to connect two containers, one having my Postgres DB, other having my EF migrations.

When I try to use the 5433 mapped port, running the migration container, I get connection refused. But if I try to connect to the PostgresDB container from an outside application, like DBeaver, I'm able to connect with no worries.

I saw this Stack Overflow Question and saw that connection between two containers in the same network don't work with mapped ports, we have to use the hosts ports, but didn't uderstand why it works like this or if it has any work arounds.

Here is my docker-compose.yml

version: '3.4'

networks:
  dev:
    driver: bridge

services:
  api:
    env_file:
      - .env
    image: avaliacaomestradoapi
    container_name: avaliacaomestradoapi-api
    depends_on:
      - postgresDB
      - migrations
    ports:
      - '${API_PORT}:80'
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      - ConnectionStrings__DefaultConnection=${CONNECTION_STRING}
      - ASPNETCORE_URLS=http://+:80
    networks:
      - dev
  
  migrations:
    env_file:
      - .env
    image: avaliacaomestrado-migrations
    container_name: avaliacaomestradoapi-migrations
    depends_on:
      - postgresDB
    build:
      context: .
      dockerfile: Dockerfile.migrations
    environment:
      - ConnectionStrings__DefaultConnection=${CONNECTION_STRING}
    networks:
      - dev
  
  postgresDB:
    env_file:
      - .env
    image: postgres:latest
    container_name: avaliacaomestradoapi-db
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DB}
    ports:
      - '5433:5432'
    restart: always
    volumes:
      - avaliacaomestrado_data:/var/lib/postgresql/data
    networks:
      - dev

volumes:
  avaliacaomestrado_data:

And here is the connection string:

Server=avaliacaomestradoapi-db;User ID=postgres;Password=postgres;Port=5433;Database=avaliacaomestrado;Integrated Security=true;Pooling=true;Maximum Pool Size=1024;Timeout=300

Here is the log:

Npgsql.NpgsqlException (0x80004005): Failed to connect to 192.168.64.2:5433
avaliacaomestradoapi-migrations  |  ---> System.Net.Sockets.SocketException (111): Connection refused
avaliacaomestradoapi-migrations  |    at Npgsql.Internal.NpgsqlConnector.Connect(NpgsqlTimeout timeout)
avaliacaomestradoapi-migrations  |    at Npgsql.Internal.NpgsqlConnector.Connect(NpgsqlTimeout timeout)
avaliacaomestradoapi-migrations  |    at Npgsql.Internal.NpgsqlConnector.RawOpen(SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt)
avaliacaomestradoapi-migrations  |    at Npgsql.Internal.NpgsqlConnector.<Open>g__OpenCore|191_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt)
avaliacaomestradoapi-migrations  |    at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
avaliacaomestradoapi-migrations  |    at Npgsql.UnpooledConnectorSource.Get(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
avaliacaomestradoapi-migrations  |    at Npgsql.NpgsqlConnection.<Open>g__OpenAsync|45_0(Boolean async, CancellationToken cancellationToken)
avaliacaomestradoapi-migrations  |    at Npgsql.NpgsqlConnection.Open()
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenDbConnection(Boolean errorsExpected)
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternal(Boolean errorsExpected)
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Open(Boolean errorsExpected)
avaliacaomestradoapi-migrations  |    at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken)
avaliacaomestradoapi-migrations  |    at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken)
avaliacaomestradoapi-migrations  |    at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists()
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.Exists()
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Migrations.Internal.Migrator.Migrate(String targetMigration)
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Design.Internal.MigrationsOperations.UpdateDatabase(String targetMigration, String connectionString, String contextType)
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Design.OperationExecutor.UpdateDatabaseImpl(String targetMigration, String connectionString, String contextType)
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Design.OperationExecutor.UpdateDatabase.<>c__DisplayClass0_0.<.ctor>b__0()
avaliacaomestradoapi-migrations  |    at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
avaliacaomestradoapi-migrations  | Failed to connect to 192.168.64.2:5433

I've tried setting the listen addresses in the postgres container with

command: postgres -c listem_addresses:'*'

using the default network in docker compose and trying to expose the 5433 mapped port with

expose:
  - 5433

but did't work.

So for now I'll stick with the solution I metioned above, using the host port, but I'm just trying to understand how it works.

Answer 1

Connections between containers use the Docker network; in your example, the custom network you've named dev. ports: only apply to connections from outside of Docker.

If you're connecting between containers in a Compose setup, the client should connect to the destination's Compose service name and whatever the "normal" port number for the target service is; for PostgreSQL, port 5432. ports: aren't required here, and they aren't considered if they're present.

A consequence of this, for example, is that you could hide your database from the host and the external network by just deleting the database's ports: block. The container wouldn't be accessible from outside of Docker, but you'd still be able to connect using the container name and the standard port number.

(I might delete all of the networks: blocks to use the default network Compose provides for you. You don't need to manually specify container_name: either. expose: is an artifact from first-generation Docker networking and does pretty much nothing, and it's safe to delete that if you do happen to have it.)