Services & Networking

We established on the Pods page that Pods are cattle: disposable, and replaced with a new IP every time. That replaceability is what makes self-healing work — but it breaks something fundamental. If Pod IPs change constantly, how does anything ever reliably talk to them? This is the fifth wall from why-orchestration, and the Service is Kubernetes’ answer.

The thread for this page: what manual, error-prone step does a Service remove — and how does it make production safer? It removes the job of hand-editing load balancer and DNS config every time a Pod restarts — the very thing that, done by hand, causes outages because someone forgets.

The pod-IP churn problem

Picture a web Deployment that needs to call an api Deployment. You scale api, a node dies, you deploy a new version — and every one of those events gives the api Pods new IPs.

   Monday:  web ──► api Pods at 10.1.4.7, 10.1.4.8, 10.1.4.9
   (a rollout happens)
   Tuesday: web ──► ??? old IPs are dead; new Pods at 10.1.6.2, 10.1.6.3, 10.1.6.4

If web hard-coded those IPs, it’s now talking to dead addresses. You could chase the changes by rewriting config on every event — but that’s a full-time, error-prone job, and the window between “Pod moved” and “you updated the config” is downtime. We need a stable address that follows the healthy Pods automatically.

The Service: one stable name, members updated for you

A Service is a named, stable virtual IP (a ClusterIP) plus a rule: route my traffic to whichever Pods currently match this label selector and are healthy. It is itself a reconciliation loop — its desired state is “send traffic to the healthy members,” and a controller keeps the membership list (the EndpointSlice) in sync as Pods come and go. You talk to the Service; the Service finds the Pods.

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: api          # any Pod with label app=api is a backend
  ports:
    - port: 80         # the Service's port
      targetPort: 8080 # the container's port

kubectl apply -f api-service.yaml
kubectl get endpointslices -l kubernetes.io/service-name=api   # the live member list
# From a Pod, just use the name — never an IP:
#   curl http://api          (same namespace)
#   curl http://api.prod.svc.cluster.local   (fully qualified)

The selector is the magic. web now calls http://api and never knows or cares which Pods answer or what their IPs are. Scale api from 3 to 30, lose a node, ship v2 — the Service’s membership updates itself, and web is none the wiser. The hand-edited LB config is gone.

How traffic actually reaches a Pod: kube-proxy

A ClusterIP is virtual — no machine literally owns it. So how does a packet to 10.96.0.10 end up at a real Pod? That’s kube-proxy, the agent on every node from the architecture page. It watches Services and EndpointSlices and programs the node’s kernel networking (iptables or, more efficiently, IPVS or the newer nftables mode) so that any packet aimed at a Service’s virtual IP is transparently rewritten to one of the healthy backend Pod IPs, load-balanced across them.

   Pod sends to:  api (10.96.0.10:80)   ← Service virtual IP
                          │
                  kube-proxy's kernel rules on the node
                          │  picks a healthy backend, rewrites destination
            ┌─────────────┼─────────────┐
            ▼             ▼             ▼
      Pod 10.1.6.2   Pod 10.1.6.3   Pod 10.1.6.4   ← real Pod IPs

Note kube-proxy doesn’t sit in the data path as a process forwarding bytes; it configures the kernel to do the rewriting. That’s why it’s fast and why a node-local outage of kube-proxy doesn’t add a hop — it just stops updating rules.

Finding Services by name: cluster DNS

The other half is DNS. The cluster runs an internal DNS server (CoreDNS) that gives every Service a name. Create a Service called api in namespace prod and it’s resolvable at api, api.prod, and api.prod.svc.cluster.local. This is exactly the service discovery idea from Part 2, made automatic: you reach dependencies by name, and the name always resolves to the current ClusterIP. Stable name → stable virtual IP → kube-proxy → current healthy Pod. Three layers, zero manual edits.

Service types: how far out the traffic comes from

ClusterIP is the default and is reachable only inside the cluster. To expose a Service more widely, you change spec.type.

Type	Reachable from	Typical use
ClusterIP (default)	Inside the cluster only	Service-to-service calls (web → api)
NodePort	Any node’s IP on a high port (30000–32767)	Quick external access, demos, dev
LoadBalancer	A cloud load balancer’s external IP	Production external access on a cloud

spec:
  type: LoadBalancer   # cloud provisions an external LB pointing at this Service
  selector: { app: web }
  ports:
    - port: 80
      targetPort: 8080

Don’t reach for LoadBalancer per service in production

Each LoadBalancer Service provisions (and bills you for) its own cloud load balancer. For many HTTP services you want one entry point doing host- and path-based routing instead. That’s an Ingress, the next page — read it before you sprinkle LoadBalancers everywhere.

A word on CNI

How do Pods get IPs and reach each other across nodes in the first place? Kubernetes doesn’t implement that itself — it defines a CNI (Container Network Interface) and you install a plugin (Calico, Cilium, Flannel, and others) that provides it. The CNI’s contract is the flat network model: every Pod gets its own cluster-wide IP, and any Pod can reach any other Pod directly, without NAT. Services, kube-proxy, and DNS all build on top of that guarantee. Different plugins add their own features — network policy, eBPF-based routing (see eBPF) — but the flat-network promise is the floor they all provide.

   Stable name (DNS) ──► Stable virtual IP (Service/ClusterIP)
        ──► kube-proxy kernel rules ──► a healthy Pod IP
                 (and the CNI is what lets Pod IPs route at all)

Under the hood — why big clusters switch kube-proxy off iptables

kube-proxy’s default iptables mode turns each Service into a chain of rules the kernel evaluates largely in sequence. That’s fine for a few hundred Services, but the rule set grows roughly linearly with Services and endpoints — so on a cluster with thousands of them, both packet matching and the time to re-sync the rules after every Pod change get painfully slow. IPVS mode (an in-kernel L4 load balancer) stores backends in hash tables instead, giving near-constant lookup and far faster updates at scale; the newer nftables mode (stable since Kubernetes v1.33) gets the same near-constant behavior from kernel map lookups, and is the recommended replacement now that IPVS mode is deprecated (v1.35). Same Service abstraction, same selector model from above — only the data-plane implementation underneath changes. The deeper networking path is in Cluster Networking Internals.

The architect’s lens

Step back from kube-proxy and DNS and judge the Service as the abstraction it is:

• Why does it exist? Because Pod IPs churn constantly — every restart, reschedule, scale, or rollout reassigns them — so a stable name and virtual IP (the ClusterIP) is the only sane way for one workload to reach another.
• What problem does it solve? It removes hand-editing load-balancer and DNS config on every Pod event: a label selector plus an EndpointSlice controller keeps the membership list current, so the caller talks to http://api and never learns which Pods answer.
• What are the trade-offs? The default iptables kube-proxy mode grows its rule set roughly linearly with Services and endpoints, so matching and re-sync slow down past a few thousand — which is why large clusters switch to IPVS or the newer nftables mode (hash/map lookup) for near-constant performance.
• When should I avoid it? Reaching for LoadBalancer per service is the anti-pattern: each one provisions and bills you for its own cloud LB with no shared TLS or URL routing — use a single Ingress for many HTTP services instead.
• What breaks if I remove it? Callers go back to hard-coding ephemeral Pod IPs that point at dead addresses after every rollout, the window between “Pod moved” and “config updated” becomes downtime, and you lose the stable-name → virtual-IP → kube-proxy → healthy-Pod chain entirely — including the flat-network floor the CNI provides underneath it.

Check your understanding

1. Why can’t web simply remember the IP addresses of the api Pods it depends on?
2. What does a Service’s label selector do, and how does that let you scale or redeploy backends without the caller noticing?
3. kube-proxy is described as not being “in the data path.” What does it do instead, and why is that faster?
4. Trace a request from curl http://api inside a Pod all the way to a backend Pod, naming DNS, ClusterIP, and kube-proxy along the way.
5. When would you choose ClusterIP vs LoadBalancer — and why is a LoadBalancer-per-service a bad default for many HTTP apps?

Show answers

1. Because Pod IPs churn constantly — every restart, reschedule, scale, or rollout gives api Pods new IPs. Hard-coded IPs would point at dead addresses, and the window between “Pod moved” and “config updated” is downtime.
2. The label selector defines which Pods are backends (any Pod matching app: api). A controller keeps the membership list (EndpointSlice) in sync as Pods come and go, so you can scale or redeploy backends freely and the caller — which talks to the stable Service name — never notices.
3. kube-proxy doesn’t forward bytes as a process; it programs the node’s kernel networking (iptables/IPVS) so any packet to a Service’s virtual IP is transparently rewritten to a healthy backend Pod. It’s faster because the rewriting happens in the kernel with no extra hop, and a local kube-proxy outage just stops updating rules rather than dropping traffic.
4. CoreDNS resolves api to the Service’s ClusterIP (a virtual IP). The packet to that ClusterIP hits kube-proxy’s kernel rules on the node, which pick a healthy backend and rewrite the destination to a real Pod IP (reachable thanks to the CNI’s flat network). Stable name → stable virtual IP → kube-proxy → current healthy Pod.
5. Use ClusterIP for in-cluster service-to-service calls (the default) and LoadBalancer to expose a Service externally on a cloud. A LoadBalancer per service is a bad default because each one provisions and bills you for its own cloud load balancer — twenty HTTP services means twenty LBs and no shared TLS or URL routing. Use one Ingress as a shared entry point instead.