Search code examples
kubernetesnetwork-programminggoogle-cloud-platformterraform-provider-gcp

Create cluster with Shared Network in GKE


I’m trying to create a cluster in GKE project-1 with shared network of project-2.

Roles given to Service account:
project-1: Kubernetes Engine Cluster Admin, Compute Network Admin, Kubernetes Engine Host Service Agent User
project-2: Kubernetes Engine Service Agent, Compute Network User, Kubernetes Engine Host Service Agent User

Service Account is created under project-1. API & Services are enabled in both Projects.

But I am getting this error persistently. Error: googleapi: Error 403: Kubernetes Engine Service Agent is missing required permissions on this project. See Troubleshooting | Kubernetes Engine Documentation | Google Cloud for more info: required “container.hostServiceAgent.use” permission(s) for “projects/project-2”., forbidden

data "google_compute_network" "shared_vpc" {
    name = "network-name-in-project-2"
    project = "project-2"
}

 
data "google_compute_subnetwork" "shared_subnet" {
    name = "subnet-name-in-project-2"
    project = "project-2"
    region = "us-east1"
}

 # cluster creation under project 1
 # project 1 specified in Provider 
resource "google_container_cluster" "mowx_cluster" {
    name = var.cluster_name
    location = "us-east1"
    initial_node_count = 1
 
    master_auth {
        username = ""
        password = ""
 
        client_certificate_config {
            issue_client_certificate = false
        }
    }
 
    remove_default_node_pool = true
    cluster_autoscaling {
        enabled = false
    }
 
    # cluster_ipv4_cidr = var.cluster_pod_cidr
    ip_allocation_policy {
        cluster_secondary_range_name = "pods"
        services_secondary_range_name = "svc"
    }
 
    network = data.google_compute_network.shared_vpc.id
    subnetwork = data.google_compute_subnetwork.shared_subnet.id
}

Solution

  • This is a community wiki answer based on the discussion in the comments and posted for better visibility. Feel free to expand it.

    The error you encountered:

    Error: googleapi: Error 403: Kubernetes Engine Service Agent is missing required permissions on this project. See Troubleshooting | Kubernetes Engine Documentation | Google Cloud for more info: required “container.hostServiceAgent.use” permission(s) for “projects/project-2”., forbidden
    

    means that the necessary service agent was not created:

    roles/container.serviceAgent - Kubernetes Engine Service Agent:

    Gives Kubernetes Engine account access to manage cluster resources. Includes access to service accounts.

    The official troubleshooting docs describe a solution for such problems:

    To resolve the issue, if you have removed the Kubernetes Engine Service Agent role from your Google Kubernetes Engine service account, add it back. Otherwise, you must re-enable the Kubernetes Engine API, which will correctly restore your service accounts and permissions. You can do this in the gcloud tool or the Cloud Console.

    The solution above works as in your use case the account was missing so it had to be (re)created.