Google Cloud Build - Access secrets of a different project during build

My project has a production and development build setup in a GCP cloud build project. These are in separate projects with separate secrets. Generally we want prod to only use prod secrets and dev to use dev secrets.

However, we'd like unit tests to run for both prod and dev and to prevent the build from completing if they fail. The unit tests always use the dev environment secrets.

Is there any way to grant the prod environments cloud build service account access to the dev environment's secrets?

Here's what I tried doing:

availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_ID/secrets/github_token/versions/latest
      env: 'GITHUB_TOKEN'
    - versionName: projects/617239722448/secrets/gpapi-gcr-env-dev/versions/latest
      env: 'DEV_TEST_ENV'

and use $$DEV_TEST_ENV in a stanza to write the secrets out to a .env.test file that the tests use (used secretEnv: ['DEV_TEST_ENV'] in the stanza).

That yields an error indicating that a specific service account didn't have access to the dev environment secrets immediately upon starting the build (didn't even run any steps).

So, I tried following these instructions to grant the service account from the error access to the dev secrets. That doesn't appear to work. I noticed in IAM that there are no security insights listed for this service account in the dev project when I granted access (unlike in the prod account where that account shows insights)...see the below screenshot for what I mean. Which makes me think it's not actually granting access.

Any ideas on how to make this work?

Solution

This was my own stupidity. The service account principal names are so close together in name. I pasted the compute service account principal into my dev project. The cloudbuild service account only shows up in the list when you check the Include Google-provided role grants checkbox as shown below. The compute one showed up in the list first and at a glance it looked the same as the cloud build service account principle.

After putting the CORRECT service account into the dev IAM project and granting it the Secret Manager Secret Accessor role, all works.