How to delete a service from all regions with gcloud run delete in cloudbuild.yaml

Question

I deploy my service on cloud run, in several regions.

My cloudbuild.yaml does this:

1. Build a new image
2. Push new image to Artifact Registry
3. TODO: Stop service in all regions
4. Run database migrations
5. Deploy service in europe-west9
6. Deploy service in europe-west8

During step 3, I want to make sure my service is stopped in all regions it is currently deployed, because I will perform migrations on the databases.

The command to delete a service is gcloud run services delete <service-name> --platform=managed --region=europe-west9. (Documentation)
And inside my cloudbuild.yaml it looks like that:

  - id: "Stop all services"
    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args: [
      'run', 'services', 'delete', "${_SERVICE_NAME}", '--platform=managed', "--region=europe-west9",
    ]

I need a way to delete the service in all regions.

I thought of doing:

1. get all regions with gcloud run regions list (Documentation).
2. iterate on result and run gcloud run services delete .....--region=... on each region.

But I have 2 issues:

1. gcloud run regions list will return all regions available in Cloud Run. So I need to filter on my ${_SERVICE_NAME}, but I don't understand how to use --filter for that purpose.
2. I have no idea how to "iterate" in my cloudbuild.yaml.

Full (simplified) cloudbuild.yaml:

steps:
  # - Build new image
  - id: "Build"
    name: "gcr.io/cloud-builders/docker"
    entrypoint: 'bash'
    args: [
      '-c',
      "docker build -t ${_ARTIFACT_REGISTRY_REPOSITORY}/${_SERVICE_NAME}:$SHORT_SHA ."
    ]

  # - Push new image to artifact registry
  - id: "Push"
    name: "gcr.io/cloud-builders/docker"
    args: ["push", "${_ARTIFACT_REGISTRY_REPOSITORY}/${_SERVICE_NAME}:$SHORT_SHA"]

  # - Stop all existing Cloud Run services (Paris, Milan, annd maybe more)
  - id: "Stop all services"
    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args: [
      'run', 'services', 'delete', "${_SERVICE_NAME}", '--platform=managed', "--region=europe-west9",
    ]
  
  # - Apply database migrations
  - id: "apply migrations"
    name: "gcr.io/google-appengine/exec-wrapper"
    entrypoint: "bash"
    args:
      [
        "-c",
        "/buildstep/execute.sh -i ${_ARTIFACT_REGISTRY_REPOSITORY}/${_SERVICE_NAME}:$SHORT_SHA -s ${_CLOUD_SQL_CONNECTION_NAME} -- bundle exec rails db:migrate"
      ]


  # - Launch paris instance
  - id: "Run Paris"
    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args: [
      'run', 'deploy', "${_SERVICE_NAME}",
      '--image', '${_ARTIFACT_REGISTRY_REPOSITORY}/${_SERVICE_NAME}:$SHORT_SHA',
      '--region', 'europe-west9',
    ]

  # - Launch Milan instance
  - id: "Run Milan"
    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args: [
      'run', 'deploy', "${_SERVICE_NAME}",
      '--image', '${_ARTIFACT_REGISTRY_REPOSITORY}/${_SERVICE_NAME}:$SHORT_SHA',
      '--region', 'europe-west8',
    ]

options:
  logging: CLOUD_LOGGING_ONLY

images:
  - "${_ARTIFACT_REGISTRY_REPOSITORY}/${_SERVICE_NAME}:$SHORT_SHA"

Answer 1

Posting as a community wiki based on the comment of @DazWilkin and @guillaumeblaquiere:

There may be a better way but you can enumerate the cloud.googleapis.com/location labels for Services with .metadata.name using gcloud run services list --project=${PROJECT} --filter="metadata.name=${_SERVICE_NAME}" --format="value(metadata.labels['cloud.googleapis.com/location'])"

Then, you can simply for REGION in $(gcloud run services list ...); do gcloud run services delete ... --region=${REGION} ...; done

Sadly, deleting a service won't stop immediately the Cloud Run instance. I recommend you to wait the Cloud Run service timeout (3 minutes by default) to be sure that all the current connections and processes are stopped. If you use CPU Always ON feature, wait 15 minutes more (in addition of the timeout). And before the migration, a database snapshot could be a good idea too in case of required rollback.

Based on the shared cloudbuild.yaml, step 3 (stop service in all regions), would look like this:

  - id: "Stop all services"
    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    script: |
      #!/usr/bin/env bash
      for REGION in $(
        gcloud run services list \
          --project=$PROJECT_ID \
          --filter="metadata.name=$SERVICE_NAME" \
          --format="value(metadata.labels['cloud.googleapis.com/location'])"
      )
      do
        echo "gcloud run services delete $SERVICE_NAME --platform=managed --region=$REGION --quiet"
        gcloud run services delete $SERVICE_NAME --platform=managed --region=$REGION --quiet
      done
    env:
      - 'SERVICE_NAME=$_SERVICE_NAME'
      - 'PROJECT_ID=$_GOOGLE_CLOUD_PROJECT_ID'

NB:

• This example doesn't implement recommended timeouts
• This example could be improved by parallelizing the ...delete... commands