Here's something interesting I am trying to figure out and looking for some help.
I am using following version
Client Version: v1.21.2
Server Version: v1.20.7
I had installed the apache solr operator using
https://solr.apache.org/operator/downloads/crds/v0.3.0/all-with-dependencies.yaml
Due to some issue I tried deleting it by removing the namespace itself. (Never do this!!)
Now I see that namespace is deleted but the custom resources are not getting deleted.
kubectl get --show-kind solrclouds.solr.apache.org -n solr-wcnp-poc --v=6
I0905 14:21:39.593361 39140 round_trippers.go:454] GET https://oidc.us-dev-aks-ad-solrpoc.k8s.us.abcd.net/apis/solr.apache.org/v1beta1/namespaces/solr-wcnp-poc/solrclouds?limit=500 200 OK in 292 milliseconds
NAME VERSION TARGETVERSION DESIREDNODES NODES READYNODES UPTODATENODES AGE
solrcloud.solr.apache.org/explore 8.8.2 1 1 0 1 7d19h
I too a help of the almighty google search and figured out that patching finalizers does the trick. So I tried the same but now seeing something interesting!
kubectl patch solrcloud.solr.apache.org/explore -p '{"metadata":{"finalizers":[]}}' -n solr-wcnp-poc --type=merge --v=6
I0905 14:39:00.028568 55762 round_trippers.go:454] GET https://oidc.us-dev-aks-ad-solrpoc.k8s.us.abcd.net/apis/solr.apache.org/v1beta1/namespaces/solr-wcnp-poc/solrclouds/explore 200 OK in 343 milliseconds
I0905 14:39:00.335421 55762 round_trippers.go:454] PATCH https://oidc.us-dev-aks-ad-solrpoc.k8s.us.abcd.net/apis/solr.apache.org/v1beta1/namespaces/solr-wcnp-poc/solrclouds/explore?fieldManager=kubectl-patch 404 Not Found in 303 milliseconds
I0905 14:39:00.338733 55762 helpers.go:216] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "namespaces \"solr-wcnp-poc\" not found",
"reason": "NotFound",
"details": {
"name": "solr-wcnp-poc",
"kind": "namespaces"
},
"code": 404
}]
F0905 14:39:00.339471 55762 helpers.go:115] Error from server (NotFound): namespaces "solr-wcnp-poc" not found
My question is, If the kubectl patch command is not able to locate the namespace then how come the kubectl get --showkind is showing the resources under the deleted namespace?
Can anybody please help me understand this?
NOTE: Now the k8s cluster is not letting me either delete the Solr-operator (the kubectl delete hangs indefinitely) like
customresourcedefinition.apiextensions.k8s.io "solrbackups.solr.apache.org" deleted
customresourcedefinition.apiextensions.k8s.io "solrclouds.solr.apache.org" deleted
customresourcedefinition.apiextensions.k8s.io "solrprometheusexporters.solr.apache.org" deleted
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io "zookeeperclusters.zookeeper.pravega.io" deleted
^C
or create a new operator (kubectl says
customresourcedefinition.apiextensions.k8s.io/solrbackups.solr.apache.org created
customresourcedefinition.apiextensions.k8s.io/solrprometheusexporters.solr.apache.org created
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
Error from server (AlreadyExists): error when creating "solr_operator.yaml": object is being deleted: customresourcedefinitions.apiextensions.k8s.io "solrclouds.solr.apache.org" already exists
Error from server (AlreadyExists): error when creating "solr_operator.yaml": object is being deleted: customresourcedefinitions.apiextensions.k8s.io "zookeeperclusters.zookeeper.pravega.io" already exists
I had the same issue and manage to delete the custom resource by recreating the namespace and run
kubectl patch solrcloud.solr.apache.org <NAME> -p '{"metadata":{"finalizers":[]}}' -n <RESTORED_NAMESPACE> --type=merge --v=6
The same solution worked with the custom resources, perconaxtradbclusters.pxc.percona.com, and zookeeperclusters.zookeeper.pravega.io that I was not able to delete