How to prevent github.com/ory/dockertest from assigning containers to random ports?

Question

I'm trying to write unit tests which run both locally using github.com/ory/dockertest and in a CircleCI environment (in which the "CI" environment variable is set) using a Docker executor type. In the container, I'd like to run the Google Pub/Sub emulator using the google/cloud-sdk image.

As a simplified example, I've written this Go program:

package main

import (
    "context"
    "flag"
    "fmt"
    "log"
    "net"
    "os"
    "time"

    "cloud.google.com/go/pubsub"
    "github.com/ory/dockertest"
    "github.com/ory/dockertest/docker"
    "google.golang.org/api/iterator"
)

var pubsubEmulatorHost string

func main() {
    flag.StringVar(&pubsubEmulatorHost, "pubsubEmulatorHost", "localhost:8085", "Google Pub/Sub emulator host")
    flag.Parse()

    if os.Getenv("CI") == "" {
        pool, err := dockertest.NewPool("")
        if err != nil {
            log.Fatalf("Could not connect to Docker: %v", err)
        }

        opts := &dockertest.RunOptions{
            Hostname:     "localhost",
            Repository:   "google/cloud-sdk",
            Cmd:          []string{"gcloud", "beta", "emulators", "pubsub", "start", "--host-port", "127.0.0.1:8085"},
            ExposedPorts: []string{"8085"},
            PortBindings: map[docker.Port][]docker.PortBinding{
                "8085/tcp": {{HostIP: "127.0.0.1", HostPort: "8085/tcp"}},
            },
        }
        resource, err := pool.RunWithOptions(opts)
        if err != nil {
            log.Fatalf("Could not start resource: %v", err)
        }

        pool.MaxWait = 10 * time.Second
        if err := pool.Retry(func() error {
            _, err := net.Dial("tcp", "localhost:8085")
            return err
        }); err != nil {
            log.Fatalf("Could not dial the Pub/Sub emulator: %v", err)
        }

        defer func() {
            if err := pool.Purge(resource); err != nil {
                log.Fatalf("Could not purge resource: %v", err)
            }
        }()
    }

    os.Setenv("PUBSUB_EMULATOR_HOST", pubsubEmulatorHost)
    defer os.Unsetenv("PUBSUB_EMULATOR_HOST")

    client, err := pubsub.NewClient(context.Background(), "my-project")
    if err != nil {
        log.Fatalf("NewClient: %v", err)
    }

    topic, err := client.CreateTopic(context.Background(), "my-topic")
    if err != nil {
        log.Fatalf("CreateTopic: %v", err)
    }
    log.Println("Created topic:", topic)

    topicIterator := client.Topics(context.Background())
    for {
        topic, err := topicIterator.Next()
        if err == iterator.Done {
            break
        }
        if err != nil {
            log.Fatalf("Next: %v", err)
        }
        fmt.Printf("%s\n", topic)
    }
}

Firstly, I've verified that running it with the CI environment variable set to a non-empty value after running the container from the command line yields the expected result:

> 
docker run -p "8085:8085" google/cloud-sdk gcloud beta emulators pubsub start --host-port=0.0.0.0:8085
Executing: /usr/lib/google-cloud-sdk/platform/pubsub-emulator/bin/cloud-pubsub-emulator --host=0.0.0.0 --port=8085
[pubsub] This is the Google Pub/Sub fake.
[pubsub] Implementation may be incomplete or differ from the real system.
[pubsub] Jul 16, 2020 9:21:33 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: IAM integration is disabled. IAM policy methods and ACL checks are not supported
[pubsub] Jul 16, 2020 9:21:34 PM io.gapi.emulators.netty.NettyUtil applyJava7LongHostnameWorkaround
[pubsub] INFO: Applied Java 7 long hostname workaround.
[pubsub] Jul 16, 2020 9:21:34 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: Server started, listening on 8085

followed by

> env CI=true go run main.go
2020/07/16 14:22:01 Created topic: projects/my-project/topics/my-topic
projects/my-project/topics/my-topic

Note that at this point, port 8085 on the container is mapped to port 8085 on the host as expected:

> docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
76724696f9d9        google/cloud-sdk    "gcloud beta emulato…"   55 seconds ago      Up 54 seconds       0.0.0.0:8085->8085/tcp   epic_ganguly

I would not like to stop the container and run the program without setting the CI environment variable, should should take care of spinning up the container automatically. What I observe, however, is that it times out trying to make a connection:

> go run main.go
2020/07/16 14:23:56 Could not dial the Pub/Sub emulator: dial tcp [::1]:8085: connect: connection refused
exit status 1

Upon inspecting the container, it seems that it is mapped to local port 32778 rather than 8085:

> docker ps
CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                     NAMES
0df07ac232d5        google/cloud-sdk:latest   "gcloud beta emulato…"   34 seconds ago      Up 33 seconds       0.0.0.0:32778->8085/tcp   wizardly_ptolemy

I would think that specifying the PortBindings in the RunOptions like they are done above should map port 8085 on the container to port 8085 on the host machine, but it seems that is not the case. Does anyone know the correct run options to make this program work?

Answer 1

Dockertest allows you to retrieve the mapped port for the container using resource.GetPort(), you can use this to set pubsubEmulatorHost to the correct value:

port := "8085"

if os.Getenv("CI") == "" {
  // ...
  pubsubEmulatorHost = opts.Hostname + resource.GetPort("8085/tcp")
  // pubsubEmulatorHost = "localhost:32778"
  // ...
}