Filters
How to search
  • Plain words find names, texts and examples.
  • "two words" finds the exact phrase.
  • --name or -x finds an option.
  • #word looks only inside commands.
  • @grep finds lessons and labs that use grep.
  • Filters: cat:network · in:labs · in:notes · level:rhcsa · ch:8 · root:yes. Put ! before a word to leave it out. OR between words means one of them.
  • ↑ ↓ to move · Enter to open · Esc to close · / to search

KubernetesNetwork series

From kubectl apply to a routable service

You run kubectl apply, the Service appears, and for a few seconds every connection is refused. The object exists; the path to a pod does not yet.

Quick reference

TimeStepWhat happens
T+0sService createdThe API server assigns a ClusterIP and writes the object to etcd.
T+1sEndpoints populatedThe controller adds ready pod IPs to an EndpointSlice.
T+2skube-proxy noticesEvery node watches Services and EndpointSlices, not just one.
T+3sRules writteniptables or IPVS rewrites the VIP to a pod IP and targetPort.
T+4sFirst packetThe client hits the VIP; conntrack remembers the rewrite.
T+5sService readyReadiness changes move the pod in and out; the rules follow.

Explained

The object

Service created. The API server validates the Service, assigns a ClusterIP from the service CIDR, and writes it to etcd. The object is real, but no node has a rule for it yet. Endpoints populated. The endpoint slice controller, inside kube-controller-manager, watches matching pods and adds ready pod IPs to an EndpointSlice. A pod that is Running but not Ready stays out.

The rules

Kube-proxy notices. Normally every node runs kube-proxy, and every kube-proxy watches Services and EndpointSlices. The update reaches all of them, not just the node running the pod. Rules written. In iptables mode, kube-proxy writes a chain that matches the ClusterIP and DNATs to a backend pod IP and targetPort. In IPVS mode it creates virtual and real servers. With no endpoints, iptables mode rejects instead.

The packet

First packet. The client connects to ClusterIP:port. The node's rules rewrite the destination to a pod IP, conntrack remembers the mapping, and the reply is rewritten back. Ready. After that, readiness changes move the pod in and out of the slice, and the rules follow. The Service never sees a packet.

The honest caveat: kube-proxy is not the only implementation. Cilium and Calico can replace it with eBPF or direct routing, so the rule order differs even when the result is the same.

The lesson: a Service is a promise the data plane has to keep.

In one line

The VIP is a rule, not a process. A Service is a promise the data plane has to keep.

Watch the slice

kubectl get endpointslice -w

More notes: Kubernetes

Open in the notes app: search, filters, copy This note on GitHub