Minio console login fails in cluster

I run Minio on a kubernetes cluster since May. Everything worked fine. Since the last action, updated ingress from Traefik to Nginx ingress, I cannot login to the Minio Console anymore.

I do not really know if this happen before or after the ingress update. But in all I think this is not the reason.

The secret is still there in the cluster and it looks well.

The common Minio login to browse the buckets works perfect. But not the Minio Console.

The pod is always writing in the pod log (Lens):

2021-11-29 22:01:17.806356 I | 2021/11/29 22:01:17 operator.go:73: the server has asked for the client to provide credentials
2021-11-29 22:01:17.806384 I | 2021/11/29 22:01:17 error.go:44: original error: invalid Login

No word about an error, but always Unauthorized inside the login screen. Anybody here with a similar problem in the past?

Solution

Solution 1:

The auth issue can be faced due to an expired apiserver-kubelet-client.crt. If it's expired, try to renew the cert and restart the apiserver.

In order to do this:

check if the cert is expired
remove expired certificates(.crt)
execute kubeadm alpha phase certs all

Note this:

# for kube-apiserver
--kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
--kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key

# for kubelet
--client-ca-file=/etc/kubernetes/pki/ca.crt

Solution 2:

While you've deployed cluster on Kubernetes before, you've should created Kubernetes manifest. You can try to delete them(service account, role, rolebinding) and create them once again:

Remove Service Account:

kubectl delete serviceaccount --namespace NAMESPACE_NAME SERVICEACCOUNT_NAME

Remove Cluter Role Binding:

kubectl delete clusterrolebinding CLUSTERROLEBINDING_NAME

Remove Minio directory:

rm -rf ./minio

Create the Service Account, Role, RoleBinding:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: minio-serviceaccount
  labels:
    app: minio

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: minio-role
  labels:
    app: minio
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  resourceNames:
  - "minio-keys"
  verbs:
  - get

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: minio-role-binding
  labels:
    app: minio
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: minio-role
subjects:
- kind: ServiceAccount
  name: minio-serviceaccount

Make sure that the Minio pods can access the Minio keys stored in the previously created Secret or create new secrets.

Run helm init command:

helm init --service-account=minio-serviceaccount

Recreate your Minio pod
Reinstall the charts