Search code examples
kubernetesgoogle-cloud-platformgoogle-kubernetes-enginegcloudkubectl

CLI command ordering for toggling between two GKE/kubectl accounts w/ two emails

Several weeks ago I asked this question and received a very helpful answer. The gist of that question was: "how do I switch back and forth between two different K8s/GCP accounts on the same machine?" I have 2 different K8s projects with 2 different emails (gmails) that live on 2 different GKE clusters in 2 different GCP accounts. And I wanted to know how to switch back and forth between them so that when I run kubectl and gcloud commands, I don't inadvertently apply them to the wrong project/account.

The answer was to basically leverage kubectl config set-context along with a script.

This question (today) is an extenuation of that question, a "Part 2" so to speak.

I am confused about the order in which I:

  • Set the K8s context (again via kubectl config set-context ...); and
  • Run gcloud init; and
  • Run gcloud auth; and
  • Can safely run kubectl and gcloud commands and be sure that I am hitting the right GKE cluster

My understanding is that gcloud init only has to be ran once to initialize the gcloud console on your system. Which I have already done.

So my thinking here is that I could be able to do the following:

# 1. switch K8s context to Project 1
kubectl config set-context <context for GKE project 1>

# 2. authenticate w/ GCP so that now gcloud commands will only hit the GCP
# resources associated with Project 1 (and GCP Account 1)
gcloud auth

# 3. run a bunch of kubectl and gcloud commands for Project/GCP Account 1

# 4. switch K8s context to Project 2
kubectl config set-context <context for GKE project 2>

# 5. authenticate w/ GCP so that now gcloud commands will only hit the GCP
# resources associated with Project 2 (and GCP Account 2)
gcloud auth

# 6. run a bunch of kubectl and gcloud commands for Project/GCP Account 2

Is my understanding here correct or is it more involved/complicated than this (and if so, why)?

Solution

  • I'll assume familiarity with the earlier answer

    gcloud

    gcloud init need only be run once per machine and only again if you really want to re-init'ialize the CLI (gcloud).

    gcloud auth login ${ACCOUNT} authenticates a (Google) (user or service) account and persists (on Linux by default in ${HOME}/.config/gcloud) and renews the credentials.

    gcloud auth list lists the accounts that have been gcloud auth login. The results show which account is being used by default (ACTIVE with *).

    Somewhat inconveniently, one way to switch between the currently ACTIVE account is to change gcloud global (every instance on the machine) configuration using gcloud config set account ${ACCOUNT}.

    kubectl

    To facilitate using previously authenticated (i.e. gcloud auth login ${ACCOUNT}) credentials with Kubernetes Engine, Google provides the command gcloud container clusters get-credentials. This uses the currently ACTIVE gcloud account to create a kubectl context that joins a Kubernetes Cluster with a User and possibly with a Kubernetes Namespace too. gcloud container clusters get-credentials makes changes to kubectl config (on Linux by default in ${HOME}/.kube/config).

    What is a User? See Users in Kubernetes. Kubernetes Engine (via kubectl) wants (OpenID Connect) Tokens. And, conveniently, gcloud can provide these tokens for us.

    How? Per previous answer

    user:
  auth-provider:
    config:
      access-token: [[redacted]]
      cmd-args: config config-helper --format=json
      cmd-path: path/to/google-cloud-sdk/bin/gcloud
      expiry: "2022-02-22T22:22:22Z"
      expiry-key: '{.credential.token_expiry}'
      token-key: '{.credential.access_token}'
    name: gcp

    kubectl uses the configuration file to invoke gcloud config config-helper --format=json and extracts the access_token and token_expiry from the result. GKE can then use the access_token to authenticate the user. And, if necessary can renew the token using Google's token endpoint after expiry (token_expiry).

    Scenario

    So, how do you combine all of the above.

    1. Authenticate gcloud with all your Google accounts
    ACCOUNT="[email protected]"
gcloud auth login ${ACCOUNT}

ACCOUNT="[email protected]"
gcloud auth login ${ACCOUNT} # Last will be the `ACTIVE` account
    1. Enumerate these
    gcloud auth list

    Yields:

    ACTIVE  ACCOUNT
        [email protected]
*       [email protected] # This is ACTIVE

To set the active account, run:
    $ gcloud config set account `ACCOUNT`
    1. Switch between users for gcloud commands

    NOTE This doesn't affect kubectl

    Either

    gcloud config set account [email protected]
gcloud auth list

    Yields:

    ACTIVE  ACCOUNT
*       [email protected] # This is ACTIVE
        [email protected]

    Or you can explicitly add --account=${ACCOUNT} to any gcloud command, e.g.:

    # Explicitly unset your account
gcloud config unset account

# This will work and show projects accessible to client1
gcloud projects list [email protected]

# This will work and show projects accessible to client2
gcloud projects list [email protected]
    1. Create kubectl contexts for any|all your Google accounts (via gcloud)

    Either

    ACCOUNT="[email protected]"
PROJECT="..." # Project accessible to ${ACCOUNT}
gcloud container clusters get-credentials ${CLUSTER} \
--ACCOUNT=${ACCOUNT} \
--PROJECT=${PROJECT} \
...

    Or equivalently using kubectl config set-context directly:

    kubectl config set-context ${CONTEXT} \
--cluster=${CLUSTER} \
--user=${USER} \

    But it avoids having to gcloud config get-clusters, gcloud config get-users etc.

    NOTE gcloud containers clusters get-credentials uses derived names for contexts and GKE uses derived names for clusters. If you're confident you can edit kubectl config directly (or using kubectl config commands) to rename these cluster, context and user references to suit your needs.

    1. List kubectl contexts
    kubectl config get-context

    Yields:

    CURRENT   NAME       CLUSTER    AUTHINFO   NAMESPACE
*         client1    a-cluster  client1
          client2    b-cluster  client2
    1. Switch between kubectl contexts (clusters*users)

    NOTE This doesn't affect gcloud

    Either

    kubectl config use-context ${CONTEXT}

    Or* you can explicitly add --context flag to any kubectl commands

    # Explicitly unset default|current context
kubectl config unset current-context

# This will work and list deployments accessible to ${CONTEXT}
kubectl get deployments --context=${CONTEXT}