Commands to switch kubectl and gcloud back and forth between two totally separate projects

Edit/Update: I should have noted, that for both of these clients/projects, I do have 2 separate Gmail accounts ([email protected] and [email protected]).

I am a freelancer that is new to K8s and GCP. I happen to be on two separate projects that just started using both of these technologies around the same time, and I am trying to manage clusters and projects from both of these clients from the same laptop. I want to be able to switch back and forth between each client and their respective K8s/GCP project and I'm looking for a list of all the kubectl and gcloud commands for doing this back and forth switchback.

So to reiterate:

Client #1
- project ABC
Client #2
- project DEF

Locally when I run kubectl config get-clusters I see:

docker-desktop
gke_client-1-proj-abc-dev
gke_client-2-proj-def-dev
minikube

Where gke_client-1-proj-abc-dev is Client #1 (project ABC)'s dev environment K8s cluster, and where gke_client-2-proj-def-dev is Client #2 (project DEF)'s dev K8s cluster.

However when I run:

gcloud projects list             
PROJECT_ID              NAME                     PROJECT_NUMBER
client-1-proj-abc       Client 1 Proj ABC Local  12345 (omitting actual project #s for brevity)
client-1-proj-abc-dev   Client 1 Proj ABC Dev    23456
client-1-proj-abc-qa    Client 1 Proj ABC QA     34567

So it appears the kubectl is correctly "connected" to both client projects, but gcloud is only logged-in/authenticated/connected to Client #1 and not Client #2.

Either way, I'm looking for the exact commands I can use so that:

when I'm "connected to" (pointing at) Client #1, all gcloud and kubectl commands are scoped to them and their project; and
when I'm "connected to" (pointing at) Client #2, all gcloud and kubectl commands are scoped to them and their project; and
there's an easy way for me to tell which client/project/cluster I'm "connected to" so that I don't inadvertently run kubectl and gcloud commands intended for one client, but actually muck up the clusters of another

Thanks in advance for any and all help.

Solution

It is understandable that you want to be diligent in keeping these clusters distinct. Whichever solution you choose, you will need to remain diligent. This is because, platform providers want to (maintain security) while reducing "friction" and so this is a common challenge for developers.

NOTE I won't cover this but a complementary approach is for you to configure e.g. bash to change e.g. color settings based on configs. So, for example, you could configure bash to change your terminal's background to green for one customer and blue for another.

By default, Kubernetes stores its cluster configuration in a YAML file (on Linux) in ${HOME}/.kube/config. You can open this file and see its structure. It contains clusters, contexts (more later), users and metadata. contexts provide a tuple that binds one cluster, to one user (and sometimes to one namespace). There's usually a current-context that points to the... current (in use)... context settings.

For Google Kubernetes Engine (GKE), user entries leverage gcloud (Google's CLI for Cloud Platform) to obtain an access-token (token-key) that is used to authenticate to the GKE cluster:

  user:
    auth-provider:
      config:
        access-token: [[redacted]]
        cmd-args: config config-helper --format=json
        cmd-path: path/to/google-cloud-sdk/bin/gcloud
        expiry: "2022-02-22T22:22:22Z"
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp

Why all the preamble?

(I've not tried this but, given the above) If you have different Google accounts (e.g. [email protected] and [email protected] for each customer, the access-token for one customer's cluster won't permit you to access the other cluster. So this provides the strongest separation.

If you have a single Google account, you will need to be somewhat diligent in switching between the contexts that represent each of the clusters. When you were initially authenticated to each cluster (possibly with gcloud container clusters get-credentials, entries were written to ${HOME}/.kube/config for both of the clusters and contexts should have been created too.

It's easiest to look at the file to see what's what. But (!) there should be at least 2 distinct contexts and their entries should marry to entries in clusters and users.

You can then simply:

kubectl use-context client-1
kubectl use-context client-2

You can explicitly specify the --context on any kubectl command too though this may become tiresome:

kubectl get deployments \
--namespace=${NAMESPACE} \
--context=${CONTEXT}

You can create your own (user-defined) contexts too but you'll need to know the name of your clusters and users:

# Use a new name
kubectl config get-contexts

# Use one of each of these values
kubectl config get-clusters
kubectl config get-users

Then:

CLUSTER="..." # One of the above results
USER="..."    # One of the above results

NAMESPACE="..." # If you wish to use a specific cluster's namespace

kubectl config set-context ${CONTEXT} \
--cluster=${CLUSTER} \
--user=${USER} \
--namespace=${NAMESPACE}

Will create the following entry in your ${HOME}/.kube/config:

contexts:
- context:
    cluster: ${CLUSTER}
    namespace: ${NAMESPACE} # This is optional
    user: ${USER}
  name: ${CONTEXT}

And you could then:

kubectl config use-context ${CONTEXT}
Switched to context "${CONTEXT}".

But, unless you query the config using kubectl config current-context, you won't have a visual clue as to which context you're using (unless you do some bash magic as noted above).

I blogged about this a long time ago. Hopefully it helps Kubernetes Engine: kubectl config