Upgrade the cluster and back up etcd

$ run as a normal user# run as root (or with sudo)

Skills in this chapter

Upgrade the cluster, back up and restore etcd

Take a node out for maintenance

cordon stops new pods on a node; drain also moves the running pods away; uncordon brings the node back.

$kubectl cordon worker1
$kubectl drain worker1 --ignore-daemonsets --delete-emptydir-data
$kubectl get nodes
worker1   Ready,SchedulingDisabled   <none>   20d   v1.32.4
$kubectl uncordon worker1

Watch out

Pods without a controller (a bare kubectl run pod) are deleted by drain and not recreated. Drain needs --force for them.

3.2

Upgrade the control plane

Upgrade kubeadm first, plan, apply, then kubelet and kubectl. One minor version per step (1.32 → 1.33).

$kubectl drain cp1 --ignore-daemonsets
#apt-mark unhold kubeadm && apt-get install -y kubeadm=1.33.1-1.1 && apt-mark hold kubeadm
#kubeadm upgrade plan
#kubeadm upgrade apply v1.33.1
#apt-mark unhold kubelet kubectl && apt-get install -y kubelet=1.33.1-1.1 kubectl=1.33.1-1.1 && apt-mark hold kubelet kubectl
#systemctl daemon-reload && systemctl restart kubelet
$kubectl uncordon cp1
$kubectl get nodes

Remember

The package repository of Kubernetes is per minor version (pkgs.k8s.io/core:/stable:/v1.33/…). Change the repository file before the upgrade.

3.3

Upgrade the worker nodes

On every worker: drain, upgrade kubeadm, kubeadm upgrade node, upgrade kubelet, restart, uncordon.

$kubectl drain worker1 --ignore-daemonsets --delete-emptydir-data
#apt-get install -y kubeadm=1.33.1-1.1
#kubeadm upgrade node
#apt-get install -y kubelet=1.33.1-1.1 kubectl=1.33.1-1.1
#systemctl daemon-reload && systemctl restart kubelet
$kubectl uncordon worker1

3.4

Back up etcd

etcdctl snapshot save writes the whole cluster state to one file. It needs the etcd endpoint and its TLS certificates.

#grep -E 'listen-client-urls|cert-file|key-file|trusted-ca-file' /etc/kubernetes/manifests/etcd.yaml
#ETCDCTL_API=3 etcdctl snapshot save /opt/backup/etcd-$(date +%F).db \
    --endpoints=https://127.0.0.1:2379 \
    --cacert=/etc/kubernetes/pki/etcd/ca.crt \
    --cert=/etc/kubernetes/pki/etcd/server.crt \
    --key=/etc/kubernetes/pki/etcd/server.key
#etcdutl snapshot status /opt/backup/etcd-2026-10-27.db --write-out=table

Exam tip

The certificate paths are in etcd.yaml. Copy them from there — never guess.

3.5

Restore etcd

Restore the snapshot into a new data folder, then point the etcd static pod to it.

#etcdutl snapshot restore /opt/backup/etcd-2026-10-27.db --data-dir /var/lib/etcd-restore
#vim /etc/kubernetes/manifests/etcd.yaml
#crictl ps | grep etcd
$kubectl get pods -A

In etcd.yaml, change the hostPath of the etcd-data volume (and --data-dir if needed) to /var/lib/etcd-restore. The kubelet restarts etcd with the old state; the API server comes back after a minute.

3.6

Renew certificates

kubeadm certificates are valid for 1 year. An upgrade renews them; you can also check and renew by hand.

#kubeadm certs check-expiration
#kubeadm certs renew all
#systemctl restart kubelet

All commands in this chapter