Error running an Docker container or docker compose with postgres, golang and Debian 11, Agora appbuilder backend

Question

I spun up a Debian 11 EC2 on AWS, and installed postgres 14.5 on it and docker and docker compose on it.I added a superuser to postgres of "admin' with a password. I created my docker-compose.yml file and a .env file.

When I try to use the docker-compose.yml file, I get:

sudo docker compose up -d
services.database.environment must be a mapping

When I build my docker container with

sudo docker build . -t tvappbuilder:latest

and then try to run it with:

sudo docker run -p 8080:8080 tvappbuilder:latest --env-file .env -it
Config Path .
4:47PM INF server/utils/logging.go:105 > logging configured fileLogging=true fileName=app-builder-logs logDirectory=./logs maxAgeInDays=0 maxBackups=0 maxSizeMB=0
4:47PM FTL server/cmd/video_conferencing/server.go:71 > Error initializing database error="pq: Could not detect default username. Please provide one explicitly"

Here are the dockers so far:

sudo docker image list
REPOSITORY     TAG       IMAGE ID       CREATED         SIZE
<none>         <none>    6e5f035abda5   18 hours ago    1.82GB
tvappbuilder   latest    6166e24a47e0   21 hours ago    21.8MB
<none>         <none>    cedcaf2facd1   21 hours ago    1.82GB
hello-world    latest    feb5d9fea6a5   12 months ago   13.3kB
golang         1.15.1    9f495162f677   2 years ago     839MB

Here is the docker-compose.yml:

version: 3.7
services:
    server:
        container_name: server
        build: .
        depends_on:
            - database
        ports:
           - 8080:8080
        environment:
            - APP_ID: $APP_ID
            - APP_CERTIFICATE: $APP_CERTIFICATE
            - CUSTOMER_ID: $CUSTOMER_ID
            - CUSTOMER_CERTIFICATE: $CUSTOMER_CERTIFICATE
            - BUCKET_NAME: $BUCKET_NAME
            - BUCKET_ACCESS_KEY: $BUCKET_ACCESS_KEY
            - BUCKET_ACCESS_SECRET: $BUCKET_ACCESS_SECRET
            - CLIENT_ID: $CLIENT_ID
            - CLIENT_SECRET: $CLIENT_SECRET
            - PSTN_USERNAME: $PSTN_USERNAME
            - PSTN_PASSWORD: $PSTN_PASSWORD
            - SCHEME: $SCHEME
            - ALLOWED_ORIGIN: ""
            - ENABLE_NEWRELIC_MONITORING: false
            - RUN_MIGRATION: true
            - DATABASE_URL: postgresql://$POSTGRES_USER:$POSTGRES_PASSWORD@database:5432/$POSTGRES_DB?sslmode=disable

    database:
        container_name: server_database
        image: postgres-14.5
        restart: always
        hostname: database
        environment: 
            - POSTGRES_USER: $POSTGRES_USER
            - POSTGRES_PASSWORD: $POSTGRES_PASSWORD
            - POSTGRES_DB: $POSTGRES_DB

Here is the Dockerfile:

## Using Dockerfile from the following post: https://medium.com/@petomalina/using-go-mod-download-to-speed-up-golang-docker-builds-707591336888

FROM golang:1.15.1 as build-env

# All these steps will be cached
RUN mkdir /server
WORKDIR /server
COPY go.mod . 
COPY go.sum .

# Get dependancies - will also be cached if we won't change mod/sum
RUN go mod download
# COPY the source code as the last step
COPY . .

# Build the binary
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -installsuffix cgo -o /go/bin/server /server/cmd/video_conferencing

# Second step to build minimal image
FROM scratch
COPY --from=build-env /go/bin/server /go/bin/server
COPY --from=build-env /server/config.json config.json

ENTRYPOINT ["/go/bin/server"]

and here is the .env file:

ENCRYPTION_ENABLED=0
POSTGRES_USER=admin
POSTGRES_PASSWORD=<correct pswd for admin>
POSTGRES_DB=tvappbuilder
APP_ID=<my real app ID>
APP_CERTIFICATE=<my real app cert>
CUSTOMER_ID=<my real ID>
CUSTOMER_CERTIFICATE=<my real cert>
RECORDING_REGION=0
BUCKET_NAME=<my bucket name>
BUCKET_ACCESS_KEY=<my real key>
BUCKET_ACCESS_SECRET=<my real secret>
CLIENT_ID=
CLIENT_SECRET=
PSTN_USERNAME=
PSTN_PASSWORD=
PSTN_ACCOUNT=
PSTN_EMAIL=
SCHEME=esports1_agora
ENABLE_SLACK_OAUTH=0
SLACK_CLIENT_ID=
SLACK_CLIENT_SECRET=
GOOGLE_CLIENT_ID=
ENABLE_GOOGLE_OAUTH=0
GOOGLE_CLIENT_SECRET=
ENABLE_MICROSOFT_OAUTH=0
MICROSOFT_CLIENT_ID=
MICROSOFT_CLIENT_SECRET=
APPLE_CLIENT_ID=
APPLE_PRIVATE_KEY=
APPLE_KEY_ID=
APPLE_TEAM_ID=
ENABLE_APPLE_OAUTH=0
PAPERTRAIL_API_TOKEN=<my real token>

According to this: https://pkg.go.dev/github.com/lib/pq I probably should not need to use pq, and instead use postgres directly, but it appears it was set up this way.

Many thanks for any pointers!

Answer 1

As per the comments there are a number of issues with your setup.

The first is the error services.database.environment must be a mapping when running docker compose up -d. This is caused by lines like - APP_ID: $APP_ID in your docker-compose.yml - use either APP_ID: $APP_ID or - APP_ID=$APP_ID as per the documentation.

A further issue is that you installed Postgres on the bare OS and are then using a postgres container. You only need to do one or the other (but if using docker you will want to use a volume or mount for the Postgres data (otherwise it will be lose when the container is rebuilt).

There are probably further issues but the above should get you started.