We have been using AWS-ECS for scheduling our containers. The most pressing problem I see with ECS, is the 'resource fragmentation'.
Say I have following task definitions/pods with respective resource requirements:
POD CPU Desired Total_CPU
- task1 1024 2 2048
- task2 512 4 2048
- task3 256 1 256
(Keeping only CPU requirements for simplicity)
If we consider VM with available CPU=2048, we need at least 3 VMs to get above services running.
Going further, for running Blue Green deployment we would theoretically need just one more VM with 2048 CPU units.
However, as the deployments happen over time, the containers will be dispersed in all available VMs. This will make the Blue Green deployment take too much time (or even fail), due to unavailability of resources.
So, for the Blue Green deployment to predictably work, we would need many more extra VMs (more $$$) in the cluster.
I would like to know whether Kubernetes provides such an feature that will consolidate the pods to keep minimum resource fragmentation.
Kubernetes does not currently reschedule pods to rebalance them across nodes, but planning has begun to include a rescheduler for this purpose in our 1.3 release in a few months.