Running Google Cloud Spanner Emulator in Azure Pipeline for Integration testing

Question

I have a go app that I am writing integration tests for using Azure Pipelines. In my Azure build .yaml:

...
    - job: Run_Go_Integration_Tests
        displayName: 'Run Go Integration Tests'
        steps:
          - script: |
              sudo add-apt-repository ppa:longsleep/golang-backports
              sudo apt update
              sudo apt install golang-go
            displayName: 'Install Go'
          - script: |
              go version
            displayName: 'Check Go Version'
          - task: GoogleCloudSdkTool@1
            inputs:
              checkLatest: true
          - script: |
              make ci-itest
            displayName: 'Running Integration Tests'

The make ci-test runs the following commands before receiving the pipeline error:

##@ Initialize Spanner
gcloud-config: ## Initialize Cloud Spanner Emulator and create emulator config
    @echo "Creating emulator configuration..."
    gcloud config configurations create emulator
    gcloud config set auth/disable_credentials true
    gcloud config set project ${DEV_SPANNER_PROJECT}
    gcloud config set api_endpoint_overrides/spanner http://localhost:9020/
    gcloud config configurations activate emulator

ci-start-spanner: ## Start Cloud Spanner Emulator for CI
    gcloud components update --quiet
    @echo "Starting cloud spanner..."
    gcloud emulators spanner start --quiet &

When the pipeline executes the command gcloud emulators spanner start I receive this error:

$ gcloud emulators spanner start --quiet

Executing: /opt/hostedtoolcache/gcloud/448.0.0/x64/bin/cloud_spanner_emulator/gateway_main --hostname localhost --grpc_port 9010 --http_port 9020 
[cloud-spanner-emulator] WARNING: proto: file "google/rpc/status.proto" is already registered
[cloud-spanner-emulator]    previously from: "google.golang.org/genproto/googleapis/rpc/status"
[cloud-spanner-emulator]    currently from:  "unknown"
[cloud-spanner-emulator] See https://protobuf.dev/reference/go/faq#namespace-conflict
[cloud-spanner-emulator] 
[cloud-spanner-emulator] WARNING: proto: file "google/rpc/status.proto" has a name conflict over google.rpc.Status
[cloud-spanner-emulator]    previously from: "google.golang.org/genproto/googleapis/rpc/status"
[cloud-spanner-emulator]    currently from:  "unknown"
[cloud-spanner-emulator] See https://protobuf.dev/reference/go/faq#namespace-conflict
[cloud-spanner-emulator] 
[cloud-spanner-emulator] WARNING: proto: message google.rpc.Status is already registered
[cloud-spanner-emulator]    previously from: "google.golang.org/genproto/googleapis/rpc/status"
[cloud-spanner-emulator]    currently from:  "unknown"
[cloud-spanner-emulator] See https://protobuf.dev/reference/go/faq#namespace-conflict
[cloud-spanner-emulator] 
[cloud-spanner-emulator] WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
[cloud-spanner-emulator] I0000 00:00:1695827664.387924    4075 emulator_main.cc:39] Cloud Spanner Emulator running.
[cloud-spanner-emulator] I0000 00:00:1695827664.387942    4075 emulator_main.cc:40] Server address: localhost:9010
ERROR: gcloud crashed (ConnectionError): HTTPConnectionPool(host='localhost', port=9020): Max retries exceeded with url: /v1/projects/local/instances?alt=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fecb45060d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

Any suggestions on how to resolve the NewConnectionError error?

Answer 1

This ended up working for me. I needed to create a docker-compose.yml containing the required cloud spanner emulator image:

This file is located under my go project ./pipelines/docker-compose.yml

version: "3.9"
services:
  gcp-cloud-spanner:
    restart: always
    image: gcr.io/cloud-spanner-emulator/emulator
    ports:
      - 9010:9010
      - 9020:9020

Then I modified my Azure pipeline code to reference the newly created Docker compose file with Cloud Spanner Emulator:

- job: Run_Go_Integration_Tests
        displayName: 'Run Go Integration Tests'
        steps:
          - script: |
              sudo add-apt-repository ppa:longsleep/golang-backports
              sudo apt update
              sudo apt install golang-go
            displayName: 'Install Go'
          - script: |
              go version
            displayName: 'Check Go Version'
          - task: GoogleCloudSdkTool@1
            inputs:
              checkLatest: true
            displayName: 'Install Google Cloud SDK'
          - task: DockerCompose@0
            inputs:
              action: Run services
              dockerComposeFile: $(Build.SourcesDirectory)/pipelines/docker-compose.yml
              buildImages: false
            displayName: 'Run Integration Test Services'
          - script: |
              make ci-itest
            displayName: 'Running Integration Tests'

Within the ci-test we can now run any of the required gcloud commands against the Docker Compose emulator. You'll need to first use the cloud emulator config, create an instance and db prior to running any tests.

Makefile:

ci-itest: gcloud-config create-spanner-instance create-spanner-db ci-cloud-spanner-migrate-up run-integration-tests-here

gcloud-config: # Create emulator configuration
    @echo "Creating emulator configuration..."
    gcloud config configurations create emulator
    gcloud config set auth/disable_credentials true
    gcloud config set project ${DEV_SPANNER_PROJECT}
    gcloud config set api_endpoint_overrides/spanner http://localhost:9020/
    gcloud config configurations activate emulator

create-spanner-instance: ## Create cloud spanner instance
    export SPANNER_EMULATOR_HOST=${SPANNER_EMULATOR_HOST} && \
    gcloud spanner instances create ${DEV_SPANNER_INSTANCE} --config=emulator \
    --description="Cloud Spanner emulator" --nodes=3

create-spanner-db: ## Create cloud spanner database
    export SPANNER_EMULATOR_HOST=${SPANNER_EMULATOR_HOST} && \
    gcloud spanner databases create ${DEV_SPANNER_DB} --instance ${DEV_SPANNER_INSTANCE}

ci-cloud-spanner-migrate-up: ## Setup DB schema
    gcloud spanner databases ddl update ${DEV_SPANNER_DB} --instance='${DEV_SPANNER_INSTANCE}' --ddl-file=./migrations/{YOUR_DDL_FILE}.ddl