Search code examples
dockerkubernetesdevopsrancherk3s

Error creating new user or granting for user permissions on Rancher

I'm having problems with creating an account on Rancher. When creating a new account I get the following error:
Internal error occurred: failed calling webhook rancherauth.cattle.io: Post https://rancher-webhook.cattle-system.svc:443/v1/webhook/validation?timeout=10s

Detail:

Internal error occurred: failed calling webhook "rancherauth.cattle.io":     
Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/validation?timeout=10s": 
dial tcp 10.43.163.117:443: connect: connection refused

I'm use Rancher version v2.5.13.

Thank you,

Peter

Solution

  • This solved the problem for me.
    Looks like deployment rancher-webhook in namespace cattle-system was removed for some reason.
    You need to go to cluster local ==> project system ==> namespace cattle-system and check that again.
    enter image description here

    If deployment rancher-webhook does not exist, you can recreate it by importing yaml file contents from another rancher (go to item Import YAML from the rancher menu - top-right corner of the image) or you have to reinstall rancher to get deployment rancher-webhook.
    This is the yaml file which I use:

    apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "2"
    meta.helm.sh/release-name: rancher-webhook
    meta.helm.sh/release-namespace: cattle-system
  generation: 2
  labels:
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:meta.helm.sh/release-name: {}
          f:meta.helm.sh/release-namespace: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/managed-by: {}
      f:spec:
        f:progressDeadlineSeconds: {}
        f:replicas: {}
        f:revisionHistoryLimit: {}
        f:selector:
          f:matchLabels:
            .: {}
            f:app: {}
        f:strategy:
          f:rollingUpdate:
            .: {}
            f:maxSurge: {}
            f:maxUnavailable: {}
          f:type: {}
        f:template:
          f:metadata:
            f:labels:
              .: {}
              f:app: {}
          f:spec:
            f:containers:
              k:{"name":"rancher-webhook"}:
                .: {}
                f:env:
                  .: {}
                  k:{"name":"NAMESPACE"}:
                    .: {}
                    f:name: {}
                    f:valueFrom:
                      .: {}
                      f:fieldRef:
                        .: {}
                        f:apiVersion: {}
                        f:fieldPath: {}
                f:image: {}
                f:imagePullPolicy: {}
                f:name: {}
                f:ports:
                  .: {}
                  k:{"containerPort":9443,"protocol":"TCP"}:
                    .: {}
                    f:containerPort: {}
                    f:name: {}
                    f:protocol: {}
                f:resources: {}
                f:terminationMessagePath: {}
                f:terminationMessagePolicy: {}
            f:dnsPolicy: {}
            f:restartPolicy: {}
            f:schedulerName: {}
            f:securityContext: {}
            f:serviceAccount: {}
            f:serviceAccountName: {}
            f:terminationGracePeriodSeconds: {}
    manager: Go-http-client
    operation: Update
    time: "2021-07-22T19:25:06Z"
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:deployment.kubernetes.io/revision: {}
      f:status:
        f:availableReplicas: {}
        f:conditions:
          .: {}
          k:{"type":"Available"}:
            .: {}
            f:lastTransitionTime: {}
            f:lastUpdateTime: {}
            f:message: {}
            f:reason: {}
            f:status: {}
            f:type: {}
          k:{"type":"Progressing"}:
            .: {}
            f:lastTransitionTime: {}
            f:lastUpdateTime: {}
            f:message: {}
            f:reason: {}
            f:status: {}
            f:type: {}
        f:observedGeneration: {}
        f:readyReplicas: {}
        f:replicas: {}
        f:updatedReplicas: {}
    manager: k3s
    operation: Update
    time: "2022-06-23T03:38:49Z"
  name: rancher-webhook
  namespace: cattle-system
  resourceVersion: "291873445"
  selfLink: /apis/apps/v1/namespaces/cattle-system/deployments/rancher-webhook
  uid: 9c9d68eb-1b0d-4371-9d02-a733c22d036c
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: rancher-webhook
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: rancher-webhook
    spec:
      containers:
      - env:
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: rancher/rancher-webhook:v0.1.4
        imagePullPolicy: IfNotPresent
        name: rancher-webhook
        ports:
        - containerPort: 9443
          name: https
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: rancher-webhook
      serviceAccountName: rancher-webhook
      terminationGracePeriodSeconds: 30

    Note: If you copied the yaml of deployment rancher-webhook file from another rancher, remove the status section of the yaml file.
    Thanks!