Spreading pods evenly across nodes within a topology domain

Question

Suppose I have a cluster like this:

zoneA:
- node1: 8 CPU
zoneB:
- node2: 4 CPU
- node3: 4 CPU

The sum of resources in zoneB equals the resources in zoneA. Therefore, if I wanted to spread pods of foo evenly across zones, I could specify a pod topology spread constraint:

spec:
  topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app: foo

This will create a topology domain for each zone, and the constraint will enforce an even spread of pods across the 2 zones.

But to the best of my knowledge (and please correct me if I'm wrong), this will not guarantee an even spread across nodes within each topology domain. For example, say there are 4 pods to schedule. To evenly spread the pods within zoneB, the placement would look like this:

node1 (8 CPU): 2 pods
node2 (4 CPU): 1 pod
node3 (4 CPU): 1 pod

However, I do not see how the above constraint would prevent a placement like this:

node1 (8 CPU): 2 pods
node2 (4 CPU): 2 pods
node3 (4 CPU): 0 pods

Is there any way to express a constraint that guarantees an even spread of pods within zoneB while maintaining the even spread across zoneA and zoneB?

Answer 1

spec:
  topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app: foo
  - maxSkew: 1
    topologyKey: kubernetes.io/hostname
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app: foo

With the above the scheduler will spread pods evenly across zones. However, because in your scenario spread 8-4-4 is fine, we can't set node whenUnsatisfiable: DoNotSchedule on the second constraint.

The scheduler will try to minimize skew, but can't guarantee it.