Search code examples
kubernetesgoogle-cloud-platformgoogle-kubernetes-engineservice-accountsgoogle-iam

Service Account Unable to Interact with GKE Cluster in Another Project

A service account ($SERVICE_ACCOUNT_A) from one Google Cloud Platform (GCP) project ($PROJECT_A) is unable to interact with a Google Kubernetes Engine (GKE) cluster ($GKE_CLUSTER_B) within another GCP project ($PROJECT_B); where:

  • $PROJECT_A is the name of the project $SERVICE_ACCOUNT_A lives within
  • $SERVICE_ACCOUNT_A is of the form some-name@some-project-name@.iam.gserviceaccount.com
  • $PROJECT_B is the name of the project the $GKE_CLUSTER_B cluster lives within
  • $GKE_CLUSTER_B is a GKE cluster name, not context, of the form: some_cluster

$SERVICE_ACCOUNT_A is unable to interact with a $GKE_CLUSTER_B despite possessing roles from $PROJECT_B containing permissions that should allow it to do so.

I.e., first I created a custom role $ROLE:

gcloud iam roles create $ROLE \
--description="$ROLE_DESCRIPTION" \
--permissions=container.clusters.get,container.clusters.list \
--project=$PROJECT_B \
--title='$ROLE_TITLE'

#=>

Created role [$ROLE].
description: $ROLE_DESCRIPTION
etag: . . .
includedPermissions:
- container.clusters.get
- container.clusters.list
name: projects/$PROJECT_B/roles/$ROLE
stage: . . .
title: $ROLE_TITLE

then I associated $ROLE, from $PROJECT_B, with $SERVICE_ACCOUNT_A:

gcloud projects add-iam-policy-binding $PROJECT_B \
--member=serviceAccount:$SERVICE_ACCOUNT_A \
--role=projects/$PROJECT_B/roles/$ROLE

#=>

Updated IAM policy for project [$PROJECT_B].
auditConfigs:
. . .

and I am able to see $ROLE under $SERVICE_ACCOUNT_A:

gcloud projects get-iam-policy $PROJECT_B \
--flatten='bindings[].members' \
--format='value(bindings.role)' \
--filter="bindings.members:${SERVICE_ACCOUNT_A}"

#=>

projects/$PROJECT_B/roles/$ROLE

with the proper permissions:

gcloud iam roles describe $ROLE \
--flatten='includedPermissions' \
--format='value(includedPermissions)' \
--project=$PROJECT_B

#=>

container.clusters.get
container.clusters.list

but still unable to get $SERVICE_ACCOUNT_A to interact with $GKE_CLUSTER_B.

Why?

Solution

  • You need to enable the Kubernetes Engine API (found here) for $PROJECT_A, even if $PROJECT_A doesn't have or need a GKE cluster.

    You can confirm this by creating a new JSON key for $SERVICE_ACCOUNT_A:

    gcloud iam service-accounts keys create \
./some-key.json \
--iam-account="${SERVICE_ACCOUNT_A}" \
--key-file-type="json"

#=>

created key [$KEY_ID] of type [json] as [./some-key.json] for [$SERVICE_ACCOUNT_A]

    activate the service account:

    gcloud auth activate-service-account \
"${SERVICE_ACCOUNT_A}" \
--key-file=./some-key.json

#=>

Activated service account credentials for: [$SERVICE_ACCOUNT_A]

    confirm it's active:

    cloud auth list
                        Credentialed Accounts
ACTIVE  ACCOUNT
        . . .
*       $SERVICE_ACCOUNT_A
        your@account.user
        . . .

To set the active account, run:
    $ gcloud config set account `ACCOUNT`

    and attempt to interact with $GKE_CLUSTER_B:

    gcloud container clusters list --project=$PROJECT_B

#=>

ERROR: (gcloud.container.clusters.list) ResponseError: code=403, message=Kubernetes Engine API has not
been used in project $PROJECT_A_ID before or it is disabled. Enable it by visiting
https://console.developers.google.com/apis/api/container.googleapis.com/overview?project=$PROJECT_A_ID
then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our
systems and retry.

    where $PROJECT_A_ID is a numeric id of the form: xxxxxxxxxxxxx.

    Visit the address returned along with the 403 above (here) and enable the Kubernetes Engine API. $SERVICE_ACCOUNT_A should now be able to interact with GKE clusters within $PROJECT_B:

    gcloud container clusters list \
--project=$PROJECT_B \
--format='value(name)

#=>

. . .
some_cluster
. . .

    including $GKE_CLUSTER_B.