I am trying to set autoscaling in my kubernetes cluster. After deploying heapster and a service for it. I am facing following error.
horizontal.go:99] Failed to reconcile nginx: failed to compute desired number of replicas based on CPU utilization for Deployment/default/nginx: failed to get CPU utilization: failed to get CPU consumption and request: failed to unmarshall heapster response: json: cannot unmarshal object into Go value of type []v1alpha1.PodMetrics
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: I0909 18:15:37.432445 27951 horizontal.go:403] Successfully updated status for nginx
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: }
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: ]
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: }
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: ]
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: }
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: }
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "memory": "1512Ki"
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "cpu": "0",
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "usage": {
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "name": "nginx",
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: {
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "containers": [
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "window": "1m0s",
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "timestamp": "2016-09-09T18:15:00Z",
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: },
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "creationTimestamp": "2016-09-09T18:15:37Z"
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "namespace": "default",
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "name": "nginx-24809263-3idn3",
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "metadata": {
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: {
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "items": [
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: "metadata": {},
Sep 09 18:15:37 dev1-132869446-3-157772786 kube-controller-manager[27951]: I0909 18:15:37.424164 27951 metrics_client.go:166] Heapster metrics result: {
Heapster is successful in pulling metrics from pod and is available on
{
"metadata": {},
"items": [
{
"metadata": {
"name": "nginx-24809263-3idn3",
"namespace": "default",
"creationTimestamp": "2016-09-10T02:57:03Z"
},
"timestamp": "2016-09-10T02:56:00Z",
"window": "1m0s",
"containers": [
{
"name": "nginx",
"usage": {
"cpu": "0",
"memory": "1400Ki"
}
}
]
}
]
}
I am running into this problem with all heapster version (heapster:canary, v1.2.0-beta.2, v1.1.0 )
Below are some more details about my environment
Kubernetes version Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.5", GitCommit:"b0deb2eb8f4037421077f77cb163dbb4c0a2a9f5", GitTreeState:"clean", BuildDate:"2016-08-11T20:29:08Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.5", GitCommit:"b0deb2eb8f4037421077f77cb163dbb4c0a2a9f5", GitTreeState:"clean", BuildDate:"2016-08-11T20:21:58Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Custom/On-Premise cluster
OS (e.g. from /etc/os-release):NAME="CentOS Linux" VERSION="7 (Core)" Kernel (e.g. uname -a): Linux dev1-132869446-3-157772786 3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 31 16:04:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
recommendation in Autoscaling (HPA) failed to get CPU consumption: cannot unmarshal object into Go value of type []v1alpha1.PodMetrics is not solving the problem
This was due to version mismatch Kubernetes 1.3.5 works with Heapster v1.1.0-beta2