DEV Community

Cover image for When Pods Start Caring About Each Other: Pod Affinity and Anti-Affinity
Siddharth More
Siddharth More

Posted on Edited on AI-assisted

When Pods Start Caring About Each Other: Pod Affinity and Anti-Affinity

TL;DR

  • Node affinity picks nodes by their labels. Pod affinity picks them by which pods are already running there.
  • Affinity pulls a pod toward pods that match a label selector. Anti-affinity pushes it away from them.
  • topologyKey decides what "together" means: the same node, the same zone, or something wider.
  • Both support required and preferred scheduling rules. The IgnoredDuringExecution variants affect scheduling; they don't automatically move already-running pods.
  • Strict anti-affinity can leave pods Pending when no eligible topology domain is available. Inter-pod rules also require more scheduler work than node affinity.

Skip around if you already know the basics:

WHY: the problem

In Part 2, a pod picked nodes by their labels. In Part 3, a node pushed pods away with a taint.

Sometimes neither of those fits, because the right placement depends on what else is running. A web app may respond faster when it sits next to its cache. Three replicas of a service don't give you much if all three land on the same node and that node goes down.

Pod affinity and anti-affinity are how you describe those relationships to the scheduler.

WHAT: the two rules and topologyKey

Think of a wedding. The guests are pods, the tables are nodes, and the scheduler is the person with the seating chart. Guests get seated one at a time as they arrive.

  • Pod affinity is "seat me with my family." You're asking to land near specific pods that are already there.
  • Pod anti-affinity is "please don't seat me with my ex." You're asking to stay away from specific pods.
  • topologyKey defines the topology domain used for the relationship. kubernetes.io/hostname means the same node; topology.kubernetes.io/zone means the same zone when that label is present and consistently set.
  • Required vs preferred is the difference between "this has to happen" and "I'd like that, but I'll cope."

Both rules pick the pods they care about with a labelSelector. Here's the cache-and-web case, where the web pod has to share a node with the cache:

# cache-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: cache
  labels:
    app: cache
spec:
  containers:
  - name: redis
    image: redis:7.4-alpine
---
# web-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: web
  labels:
    app: web
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - cache
        topologyKey: kubernetes.io/hostname
  containers:
  - name: nginx
    image: nginx:1.28-alpine
Enter fullscreen mode Exit fullscreen mode

Anti-affinity has its own field, podAntiAffinity, and works the same way. This Deployment refuses to put two of its own replicas on the same node:

# spread-web.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: spread-web
spec:
  replicas: 2
  selector:
    matchLabels:
      app: spread-web
  template:
    metadata:
      labels:
        app: spread-web
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - spread-web
            topologyKey: kubernetes.io/hostname
      containers:
      - name: nginx
        image: nginx:1.28-alpine
Enter fullscreen mode Exit fullscreen mode

Both of those use required. The preferred version wraps the rule in a podAffinityTerm and adds a weight from 1 to 100, just like preferred node affinity from Part 2. This one prefers to keep matching web pods in different zones, but doesn't fail if it can't. weight: 100 gives this preference the maximum weight for this term; it does not guarantee zone spreading:

apiVersion: v1
kind: Pod
metadata:
  name: web-zone-spread
  labels:
    app: web
spec:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values:
              - web
          topologyKey: topology.kubernetes.io/zone
  containers:
  - name: nginx
    image: nginx:1.28-alpine
Enter fullscreen mode Exit fullscreen mode

A few details worth knowing:

  • The labelSelector supports In, NotIn, Exists, and DoesNotExist. There's no Gt or Lt here like there is for node affinity.
  • By default, a rule only looks at pods in the same namespace as the pod being scheduled. You can widen that with namespaces or namespaceSelector.
  • topologyKey is required. Choose a node label that exists consistently across the topology domains you intend to compare. kubernetes.io/hostname is normally set on nodes; zone labels depend on the cluster and infrastructure.

📖 Docs: Inter-pod affinity and anti-affinity

WHEN: where this actually shows up

A few places this comes up:

  • Putting an app next to its cache or database to cut latency, or to avoid cross-zone network costs
  • Spreading replicas across nodes, so one node failing doesn't take the whole service with it
  • Spreading replicas across zones, for the same reason one level up
  • Keeping two heavy workloads off the same node when they'd get in each other's way

If none of that sounds like your situation, the scheduler's default placement is usually fine.

HOW: what's actually happening

Required rules get checked during filtering, and preferred ones during scoring, same as everything else so far.

What's different is what the scheduler has to look at. Node affinity evaluates node labels. Pod affinity and anti-affinity also consider matching pods and the topology domains of candidate nodes, which can add scheduler work. Kubernetes cautions against using inter-pod affinity and anti-affinity in clusters larger than several hundred nodes. Treat that as guidance rather than a hard cutoff: the impact depends on cluster size, workload, and scheduling patterns.

Because scheduling decisions depend on pods that are already placed, order matters. Back at the wedding: if your family hasn't arrived yet, there's nobody to sit with, so you wait. A pod with required affinity to a cache that isn't running yet may stay Pending until a matching pod is running and the other scheduling requirements can also be satisfied.

An important nuance: a required anti-affinity rule on an existing pod can prevent a new pod from scheduling even when the new pod has no anti-affinity rule of its own. This depends on whether the existing rule's selector matches the incoming pod, and on the rule's namespace scope and topology domain. Preferred anti-affinity influences scoring; it does not impose the same hard restriction. Your ex asked not to share a table with you, so that table is off the list if the rule actually applies to you.

DIFFERENCE: pod affinity vs node affinity

Node affinity is about what a node is. It has an SSD, or it's in a certain zone. Pod affinity is about who's already there. At the wedding, that's the difference between "seat me at a table by the window" and "seat me with my family."

That connects the last few parts. Node affinity pulls a pod toward a kind of node. Taints keep pods off a node unless they tolerate it. Pod affinity and anti-affinity react to the neighbors.

One more link back to Part 2: node anti-affinity isn't its own field, you get it from NotIn or DoesNotExist. Pod anti-affinity does have its own field, podAntiAffinity.

If your main goal is to distribute replicas evenly across nodes or zones, topology spread constraints are often a better fit because they express distribution directly. They're a separate feature, so they're outside this article; see the official topology spread constraints documentation.

FAILURE: what goes wrong

The classic case is required anti-affinity on kubernetes.io/hostname with more replicas than there are eligible nodes. If each eligible node already has a matching replica, an additional replica has nowhere to go and stays Pending. This assumes the selector matches the replicas, no other eligible nodes are available, and the existing pods and nodes satisfy the other scheduling requirements. The FailedScheduling event commonly mentions pod anti-affinity, though the exact wording varies by Kubernetes version.

Rolling updates can hit the same wall. During an update, a new pod may match the anti-affinity selector on old pods. If every eligible node already has a matching pod, the new pod may not schedule and the rollout can stall. Kubernetes supports matchLabelKeys to incorporate selected labels from the incoming pod into the affinity selector; pod-template-hash can be used to distinguish Deployment revisions. This feature is stable in Kubernetes v1.33. However, it is not a universal fix: existing old pods' own required anti-affinity rules are still evaluated and may continue to block new pods. Check the behavior against your Kubernetes version and rollout configuration. Switching to preferred anti-affinity removes the hard placement restriction, but also removes the guarantee that replicas stay apart.

Then there's the topology key. Use a topology label that is present and consistent across the nodes you expect the scheduler to compare. kubernetes.io/hostname is normally set on nodes, while topology.kubernetes.io/zone depends on the cluster and infrastructure and may be absent in local or kubeadm clusters. Missing or inconsistent topology labels can cause unexpected scheduling behavior, particularly with anti-affinity.

Namespaces catch people too. By default, a pod affinity term matches pods in the same namespace as the incoming pod. Use namespaces or namespaceSelector when you need a different scope.

The IgnoredDuringExecution suffix means these rules are not continuously enforced after scheduling. If pod or node labels change later, Kubernetes does not automatically evict or relocate an already-running pod just because the affinity relationship is no longer satisfied.

PRACTICE: deploy, scale, check

For the anti-affinity exercise, use at least two eligible worker nodes. The cache-and-web affinity example can run on one node, provided a matching cache pod is running and there are enough resources.

Start with affinity. Save the two pods from the WHAT section as cache-pod.yaml and web-pod.yaml, then apply them:

kubectl apply -f cache-pod.yaml
kubectl apply -f web-pod.yaml
kubectl get pods -o wide
Enter fullscreen mode Exit fullscreen mode

If the cache pod is running and the cluster has enough resources, the web pod should schedule on the same node as the cache. Now remove them and start only the web pod:

kubectl delete pod cache web
kubectl apply -f web-pod.yaml
kubectl get pods
Enter fullscreen mode Exit fullscreen mode

With required affinity and no matching cache pod running, the web pod should remain Pending. The Events section helps explain why:

kubectl describe pod web
Enter fullscreen mode Exit fullscreen mode

Look for a FailedScheduling event that mentions pod affinity. Apply cache-pod.yaml again and the web pod should get scheduled next to it.

Now anti-affinity. Save the Deployment as spread-web.yaml, apply it, and check that the replicas landed on different nodes:

kubectl apply -f spread-web.yaml
kubectl get pods -o wide
Enter fullscreen mode Exit fullscreen mode

Then scale it past your worker node count. If you have two workers, try five:

kubectl scale deployment spread-web --replicas=5
kubectl get pods
Enter fullscreen mode Exit fullscreen mode

If those two workers are the only eligible nodes, each can host at most one matching replica under this required anti-affinity rule. The remaining replicas should stay Pending. Use kubectl describe pod on a pending replica to inspect its FailedScheduling event. Clean up when you're done:

kubectl delete -f cache-pod.yaml -f web-pod.yaml -f spread-web.yaml --ignore-not-found
Enter fullscreen mode Exit fullscreen mode

Next up, Part 5: DaemonSets, where a pod gets placed on every node, and the taints from Part 3 show up again.

If anything here didn't land, or you'd explain it differently, drop a comment. Doubts and pushback are both welcome.

Top comments (0)