I'm using gsutil rsync
in my Jenkins instance for deploying code to composer and I'd like to be able to deploy code to different projecs (prodcution, staging, dev...). When using gcloud
the only thing that I need to do is to provide the --account
parameter in order to pick the service account that allows Jenkins to do that but seems like gsutils
only works with config files and that creates a race condition when several jobs are running simultaneously because it will all depend on the configuration present in gcloud config
.
Is there a way to specify which account must be used by Google Cloud's gsutil
?
First of all, note that if you're using an installation of gsutil bundled with gcloud, gcloud will pass its currently active credentials to gsutil. If you want to avoid this and use multiple different credentials/accounts for overlapping invocations, you should manage credentials via gsutil directly (using separate boto config files), not gcloud. You can disable gcloud's auto-credential-passing behavior via running gcloud config set pass_credentials_to_gsutil false
.
Separate gsutil installations will all write to the same state directory by default ($HOME/.gsutil
), as well as loading the same default boto config files. To avoid race conditions, you can (and should) use the same gsutil installation, but specify a different state_dir
and/or boto config file for invocations that might overlap. This can be set either at the boto config file level, or with the -o
option, e.g. gsutil -o "GSUtil:state_dir=$HOME/.gsutil2" cp src dst
. You'll find more information about it here.