Search code examples
javascriptamazon-web-serviceskuberneteskubectlamazon-eks

How to connect - correctly authenticated - to EKS with loadFromOptions using @kubernetes/client-node?


I am trying to use @kubernetes/client-node to access a kubernetes API running on AWS EKS. I have setup a new IAM user which is allowed to access the Kubernetes API (eks:AccessKubernetesApi).

This here is an excerpt from my code, I am mostly confused on how to provide the user credentials (since in the kube config they would be provided by exec, and I'm unsure what that resolves to).

const kubeConfigData = await getEksKubeConfigData(); // this gives me all clusters with relevant EKS data included

const clusters = kubeConfigData.map((cluster) => ({
  name: cluster.arn as string,
  server: cluster.endpoint as string,
  caData: cluster.certificateAuthority as string,
  skipTLSVerify: false,
}));

const contexts = kubeConfigData.map((cluster) => ({
  name: cluster.arn as string,
  cluster: cluster.arn as string,
  user: cluster.arn as string,
}));

/** 
As far as I understand here lies the problem.
I am unsure how to correctly authenticate against the api, can I provide the token here?
The access id and secret? 
I can't read a kube config from the filesystem, so I need to provide it either via STS token or through env variables, as far as I understand?
*/
const users = kubeConfigData.map((cluster) => ({
  name: cluster.arn as string,
  password: cluster.token as string,
}));

const currentContext = contexts[0].name;

kubeConfig.loadFromOptions({
  clusters,
  contexts,
  users,
  currentContext,
});

Trying to listNamespace() with this config results in the following response body:

body: {
    kind: 'Status',
    apiVersion: 'v1',
    metadata: {},
    status: 'Failure',
    message: 'namespaces is forbidden: User "system:anonymous" cannot list resource "namespaces" in API group "" at the cluster scope',
    reason: 'Forbidden',
    details: { kind: 'namespaces' },
    code: 403
  }

Please tell me what I'm doing wrong.


Solution

  • Ok, so my main problem and the reason why I thought I needed to use loadFromOptions was that I had missing RBAC permissions and didn't have a connection between my IAM user and RBAC.

    Here is what I did if anyone finds this and wants to know:

    First things first: create a RBAC role by applying the following yaml, using the original account that created the EKS cluster. Here I gave it the name rbac-add-reader.yaml, while giving read acces to all resources to the new role reader:

    kubectl apply -f rbac-add-reader.yaml

    ---  
    apiVersion: rbac.authorization.k8s.io/v1  
    kind: ClusterRole  
    metadata:  
    name: reader  
    rules:  
    - apiGroups: ["*"]  
    resources: ["*"]  
    verbs: ["get", "list", "watch"]  
    ---  
    apiVersion: rbac.authorization.k8s.io/v1  
    kind: ClusterRoleBinding  
    metadata:  
    name: reader  
    subjects:  
    - kind: Group  
    name: reader  
    apiGroup: rbac.authorization.k8s.io  
    roleRef:  
    kind: ClusterRole  
    name: reader  
    apiGroup: rbac.authorization.k8s.io
    

    Create a new IAM User. Create an IAM policy (read access to all eks properties, as well as the ability to get a session token):

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "eks:DescribeNodegroup",
                    "eks:ListNodegroups",
                    "eks:DescribeCluster",
                    "eks:ListClusters",
                    "eks:AccessKubernetesApi",
                    "eks:ListUpdates",
                    "eks:ListFargateProfiles",
                    "sts:GetSessionToken",
                    "ssm:GetParameter",
                    "eks:DescribeFargateProfile",
                    "eks:DescribeAddonConfiguration",
                    "eks:ListTagsForResource",
                    "eks:ListAddons",
                    "eks:DescribeAddon",
                    "eks:DescribeIdentityProviderConfig",
                    "eks:DescribeUpdate",
                    "eks:DescribeAddonVersions",
                    "eks:ListIdentityProviderConfigs"
                ],
                "Resource": "*"
            }
        ]
    }
    

    Create a user group, add the policy to the group and then add the newly created user to the group too.

    Now we need to connect the IAM user to the RBAC role (if you don't want to use vim prepend KUBE_EDITOR="nano"): kubectl edit -n kube-system configmap/aws-auth

    Add the IAM user to mapUsers:

    apiVersion: v1
    data:
      mapAccounts: |
        []
      mapRoles: |
        []
      mapUsers: |
        - "userarn": "arn:aws:iam::XXXXXXXXXXX:user/<IAM-user-name>"
          "username": "<IAM-user-name>"
          "groups":
          - "reader"
    kind: ConfigMap
    .
    .
    .
    
    

    Switch over to your new user. (If you have not, configure a new profile with aws configure --profile <your-new-profile-name>)

    Update your kubeConfig: aws eks update-kubeconfig --name <clustername> --profile <your-new-profile-name>

    You can now check if your permissions were set correctly: kubectl auth can-i get pods should return yes, kubectl auth can-i create pods should return no.

    Now you are able to loadFromDefault().