Error: namespaces is forbidden: User "system:serviceaccount:default:test" cannot create resource "namespaces" in API group ""

Question

I want to configure native Kubernetes cluster using Terraform script. I tried this Terraform script:

terraform {
  required_providers {
    kubernetes = {
      source = "hashicorp/kubernetes"
      version = "2.13.1"
    }
    kubectl = {
      source = "gavinbunney/kubectl"
      version = "1.14.0"
    }
    helm = {
      source = "hashicorp/helm"
      version = "2.6.0"
    }
  }
}

provider "kubectl" {
  # run kubectl cluster-info to get expoint and port
  host = "https://192.168.1.139:6443/"
  token = "eyJhbGciOiJSUzI1NiIsImt....."
  insecure = "true"
}

provider "kubernetes" {
  # run kubectl cluster-info to get expoint and port
  host = "https://192.168.1.139:6443/"
  token = "eyJhbGciOiJSUzI1NiIsImt....."
  insecure = "true"
}

resource "kubernetes_namespace" "example" {
  metadata {
    annotations = {
      name = "example-annotation"
    }

    labels = {
      mylabel = "label-value"
    }

    name = "terraform-example-namespace"
  }
}

ref: https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs

I tried to create a user from this tutorial: https://killercoda.com/kimwuestkamp/scenario/k8s1.24-serviceaccount-secret-changes

kubectl create sa cicd

kubectl get sa,secret


cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: cicd
spec:
  serviceAccount: cicd
  containers:
  - image: nginx
    name: cicd
EOF




kubectl exec cicd -- cat /run/secrets/kubernetes.io/serviceaccount/token && echo

kubectl exec cicd cat /run/secrets/kubernetes.io/serviceaccount/token && echo


kubectl create token cicd

kubectl create token cicd --duration=999999h



cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  name: cicd
  annotations:
    kubernetes.io/service-account.name: "cicd"
EOF



kubectl get sa,secret

kubectl describe secret cicd

kubectl describe sa cicd

kubectl get sa cicd -oyaml

kubectl get sa,secret

When I run the Terraform script I get error:

kubernetes_namespace.example: Creating...
╷
│ Error: namespaces is forbidden: User "system:serviceaccount:default:cicd" cannot create resource "namespaces" in API group "" at the cluster scope
│
│   with kubernetes_namespace.example,
│   on main.tf line 36, in resource "kubernetes_namespace" "example":
│   36: resource "kubernetes_namespace" "example" {

Can you advise what user configuration I'm missing?

Can you advise what is the proper way to implement this script and provision HELM chart into native Kubernetes.

Answer 1

Error: namespaces is forbidden: User "system:serviceaccount:default:cicd" cannot create resource "namespaces" in API group "" at the cluster scope

The service account cicd in namespace default is lacked of permissions. You can first assign cluster-admin permissions to ensure your pipeline is functioning, then trim the permissions gradually according to your use case. Apply the following spec before your pipeline starts:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: <of your own>
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: cicd
  namespace: default