Control where pods run and how much they use

$ run as a normal user# run as root (or with sudo)

Skills in this chapter

Resource limits, taints, affinity and pod admission

Set requests and limits

Requests reserve resources (the scheduler uses them). Limits cap them (CPU is throttled, memory over the limit is killed).

resources (container part)

resources:
  requests:
    cpu: 250m
    memory: 256Mi
  limits:
    cpu: "1"
    memory: 512Mi

$kubectl set resources deployment web --requests=cpu=250m,memory=256Mi --limits=cpu=1,memory=512Mi -n shop
$kubectl describe node worker1 | grep -A6 "Allocated resources"
$kubectl top nodes

Remember

250m = a quarter of a CPU. A pod stays Pending when no node has enough unrequested resources.

7.2

Limit a namespace

LimitRange sets defaults and maximums per container. ResourceQuota caps the total of a namespace.

limits.yaml

apiVersion: v1
kind: LimitRange
metadata: { name: defaults, namespace: shop }
spec:
  limits:
    - type: Container
      default: { cpu: 500m, memory: 512Mi }
      defaultRequest: { cpu: 100m, memory: 128Mi }
      max: { cpu: "2", memory: 2Gi }
---
apiVersion: v1
kind: ResourceQuota
metadata: { name: team-quota, namespace: shop }
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    pods: "30"

$kubectl create quota team-quota --hard=pods=30,requests.cpu=4 -n shop
$kubectl describe quota -n shop

7.3

Choose nodes with selectors and affinity

nodeSelector is a simple label match. Node affinity adds rules ("required" or "preferred"). Pod anti-affinity spreads copies over nodes.

$kubectl label node worker2 disk=ssd

affinity (pod spec part)

nodeSelector:
  disk: ssd
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - { key: topology.kubernetes.io/zone, operator: In, values: [zone-a, zone-b] }
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector: { matchLabels: { app: web } }
          topologyKey: kubernetes.io/hostname

7.4

Keep pods away with taints

A taint on a node repels pods. Only pods with a matching toleration may run there.

$kubectl taint nodes worker3 gpu=true:NoSchedule
$kubectl taint nodes worker3 gpu=true:NoSchedule-
$kubectl describe node cp1 | grep Taints
Taints:             node-role.kubernetes.io/control-plane:NoSchedule

toleration (pod spec part)

tolerations:
  - key: gpu
    operator: Equal
    value: "true"
    effect: NoSchedule

Effect	Meaning
NoSchedule	New pods without toleration are not placed here
PreferNoSchedule	Avoid if possible
NoExecute	Also evict running pods without toleration

Remember

A toleration allows a pod on a tainted node; it does not force it there. Combine with node affinity to force.

7.5

Run static pods

A static pod is a YAML file in the kubelet's manifest folder. The kubelet runs it directly, without the scheduler or the API server.

#grep staticPodPath /var/lib/kubelet/config.yaml
staticPodPath: /etc/kubernetes/manifests
#kubectl run static-web --image=nginx --dry-run=client -o yaml > /etc/kubernetes/manifests/static-web.yaml
$kubectl get pods -A | grep static-web

The pod appears with the node name as suffix (static-web-worker1). Delete it by removing the file.

7.6

Use priority classes

When resources are short, pods with a higher PriorityClass are scheduled first and can push lower ones out.

$kubectl create priorityclass critical-apps --value=100000 --description="Business critical"
$kubectl get priorityclass

All commands in this chapter