I am trying to execute below commands in a Kubeflow(v1.4.1) Jupyter Notebook.
KServe = KServeClient()
KServe.create(isvc)
I am getting mentioned error while attempting to execute above mentioned command.
ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': '86bb1b59-20ae-4127-9732-d0355671b12f', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': 'a5a5d542-8a9a-4031-90d9-4faf01914391', 'X-Kubernetes-Pf-Prioritylevel-Uid': '6846984d-14c5-4f4d-9251-fe97d91b17fc', 'Date': 'Thu, 02 Jun 2022 07:53:30 GMT', 'Content-Length': '429'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"inferenceservices.serving.kserve.io is forbidden: User \"system:serviceaccount:kubeflow-user-example-com:default-editor\" cannot create resource \"inferenceservices\" in API group \"serving.kserve.io\" in the namespace \"kubeflow-user-example-com\"","reason":"Forbidden","details":{"group":"serving.kserve.io","kind":"inferenceservices"},"code":403}
As a mitigation step I have added underlying manifests to my Kubernetes cluster via kubectl apply -f <manifest-location.yaml>
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: sla-manager-service-role
namespace: default //(have tried it with kubeflow-user-example-com as well)
labels:
app: sla-manager-app
rules:
- apiGroups: ["serving.kserve.io"] # "" indicates the core API group
resources: ["inferenceservices"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: sla-manager-service-role
subjects:
- kind: ServiceAccount
name: default //(have added it with default-editor as well)
namespace: kubeflow-user-example-com
But this does not resolved the issue.
Output I am receiving for
kubectl auth can-i create inferenceservice --as=system:serviceaccount:kubeflow-user-example-com:default-editor -n default
(Output) no
kubectl auth can-i create inferenceservices
(Output) yes
Can you please help me with what I am missing here?
Got to know that KServe is not comaptible with Kubeflow version 1.4 and works for Kubeflow version >=1.5.
Switching to kfserving 0.6 resolved my issue.