I am trying to get my deployment to only deploy replicas to nodes that aren't running rabbitmq (this is working) and also doesn't already have the pod I am deploying (not working).
I can't seem to get this to work. For example, if I have 3 nodes (2 with label of app.kubernetes.io/part-of=rabbitmq) then all 2 replicas get deployed to the remaining node. It is like the deployments aren't taking into account their own pods it creates in determining anti-affinity. My desired state is for it to only deploy 1 pod and the other one should not get scheduled.
kind: Deployment
metadata:
name: test-scraper
namespace: scrapers
labels:
k8s-app: test-scraper-deployment
spec:
replicas: 2
selector:
matchLabels:
app: testscraper
template:
metadata:
labels:
app: testscraper
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/part-of
operator: In
values:
- rabbitmq
- key: app
operator: In
values:
- testscraper
namespaces: [scrapers, rabbitmq]
topologyKey: "kubernetes.io/hostname"
containers:
- name: test-scraper
image: #######:latest```
I think Thats because of the matchExpressions part of your manifest , where it requires pods need to have both the labels app.kubernetes.io/part-of: rabbitmq and app: testscraper to satisfy the antiaffinity rule.
Based on deployment yaml you have provided , these pods will have only app: testscraper but NOT pp.kubernetes.io/part-of: rabbitmq hence both the replicas are getting scheduled on same node
from Documentation (The requirements are ANDed.):
kubectl explain pod.spec.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution.labelSelector
...
FIELDS:
matchExpressions <[]Object>
matchExpressions is a list of label selector requirements.
**The requirements are ANDed.**