How to set cooldown period for the Kops cluster autoscaler?

The autoscaler plugin works great, but it defaults to a 10 minute cooldown for unneeded nodes. Since AWS has per-second billing, we'd like to scale down faster to reduce cost.

Is the cooldown period configurable in the kops InstanceGroup or the cluster autoscaler?

Solution

You need to apply --scale-down-unneeded-time flag while defining autoscaler in deployment configuration file:

Here is an example:

 command:
   - ./cluster-autoscaler
   - --cloud-provider=aws
   - ...
   - --scan-interval=25s
   - --scale-down-unneeded-time=30s

It is useful to add also --scan-interval flag.

Also remember that short cooldowns can cause Over-Scaling / Under-Scaling.

Cooldown durations should instead be at least the amount of time it takes the target metric to reach its new “normal” after a scaling event. If it takes three minutes for your CPU utilization to drop by about 50% after scaling up 2x, a cooldown less than three minutes causes AWS to scale out again before the previous scale out has had time to take effect on your metrics, causing it to scale out more than necessary.

Take a look: autoscaling-aws, kubernetes-autoscaler, autoscaler-aws-kubernetes.