Service Discovery

We’ve reached the last and most modern piece of the networking story. A load balancer needs a list of healthy backends to send traffic to. But where does that list come from when instances are appearing, dying, and being rescheduled onto different machines every few minutes? Service discovery is the answer to “where is service X right now?” — and it’s the idea that makes containers and Kubernetes possible at all.

The problem: services that move

In the old world, you ran the orders service on a known box with a fixed IP, and every client just hard-coded that address. It worked because nothing moved. That world is gone. In a cloud or container environment:

• Instances are created and destroyed constantly (autoscaling, deploys, crashes).
• Each new instance gets a different IP than the one it replaced.
• A container can be rescheduled onto a different host at any moment.
• You run many instances of each service, and the set changes minute to minute.

   What the "orders" service looks like over five minutes:

   10:00   10.0.1.5   10.0.1.8   10.0.1.9
   10:02   10.0.1.5              10.0.1.9   10.0.2.3   (one died, one was added)
   10:04              10.0.1.8   10.0.1.9   10.0.2.3   (5 replaced by 8 elsewhere)

Hard-coding any of those addresses is a guaranteed future outage. This is the fallacy “topology doesn’t change,” and at container scale it’s violated continuously. Service discovery makes the question dynamic: clients ask “give me a healthy orders instance” and get a current answer.

The shape of a solution

Every discovery system has the same three moving parts. Hold these and the rest is detail:

   1. REGISTER   an instance starts -> announces "I am orders, at 10.0.2.3:8080"
   2. HEALTH     the system keeps checking it's still alive; dead ones drop out
   3. LOOKUP     a client asks "where is orders?" -> gets the current healthy set

The magic is that registration and de-registration happen automatically as instances come and go, and health-checking (the same idea from the load balancer page) ensures lookups never return a corpse. No human edits a list. There are two common ways to build this.

Approach 1: DNS-based discovery

Reuse the phone book you already understand. Instead of a static record, the service name resolves to the current set of healthy instances, kept up to date automatically:

   client -> "orders.internal?"  -> [ DNS ] -> 10.0.1.8, 10.0.2.3  (today's healthy set)

This is elegant because every language already knows how to resolve a name — no special client library needed. It’s exactly how Kubernetes does it: a Service named orders becomes the resolvable name orders.default.svc.cluster.local, and the cluster keeps that name pointing at the live pods behind it. The catch is DNS caching: TTLs and client-side caches can serve stale answers, so discovery DNS uses very short TTLs and the platform updates records aggressively.

Approach 2: registry-based discovery

Run a dedicated, strongly-consistent registry — a service whose whole job is to track services. Instances register with it directly; clients (or a sidecar proxy) query it:

   orders instance --register--> [ Registry ]  <--query-- client
                                  (Consul,                "healthy orders?"
                                   etcd, Eureka,          -> 10.0.1.8, 10.0.2.3
                                   ZooKeeper)

Tools like Consul, etcd, Eureka, and ZooKeeper fill this role. A registry can hold richer metadata than DNS (version, region, load, custom tags), push updates instantly instead of waiting for a TTL, and make smarter routing decisions. The cost is more moving parts and a client or sidecar that knows how to talk to it. Notably, Kubernetes uses etcd as the registry for cluster state under the hood, then exposes discovery to your apps through the friendlier DNS interface above — the two approaches working together.

Health checks tie it together

Both approaches live or die by health checks — the same automated loop from load balancing, now load bearing for the entire topology. An instance that fails its check is removed from the registry or DNS record, so it stops receiving traffic within seconds:

   register -> [ healthy: receives traffic ]
                     |
                  fails health check
                     v
               removed from discovery -> stops receiving traffic
                     |
                  recovers
                     v
               re-registered -> receives traffic again

This closes the loop: discovery isn’t just “where are the instances,” it’s “where are the healthy instances, right now.” That distinction is what lets a system route around failure without anyone noticing.

Why this underpins orchestration

Step back and look at what we’ve built across this whole part. Service discovery is the keystone:

• Load balancers need it to know their backend list.
• Autoscaling is only possible because new instances auto-register and start receiving traffic with no manual wiring.
• Self-healing works because a crashed instance silently drops out of discovery and its replacement drops in.
• Zero-downtime deploys roll new instances in and old ones out through discovery.

Kubernetes is, from this angle, a giant orchestrated loop built on service discovery: it schedules containers onto whatever hosts have room, gives each Service a stable discoverable name, health-checks the pods behind it, and keeps the discovery records matching reality — forever, automatically. Everything in Part 4 assumes this page.

The thread

The manual, error-prone step service discovery removes is maintaining the map by hand — the config files of IP addresses, the load-balancer member lists someone edited after every deploy, the hard-coded endpoints that broke the moment a box was replaced. Discovery makes the map a live, self-updating, health-checked query. That’s what makes production safe at scale: instances are free to come and go, fail and recover, scale up and down — and the system always knows where to send the next request, with no human keeping the list current. It’s the bridge from “servers you tend” to “infrastructure that tends itself.”

Under the hood — client-side vs server-side discovery

DNS-based and registry-based describe where the list lives. There’s a second axis: who picks the instance. In client-side discovery, the client itself queries the registry, gets the full healthy set, and runs its own load-balancing logic to choose one (Netflix’s Eureka with a smart client library is the classic example). It’s efficient — no extra network hop — but every client, in every language, needs the discovery logic baked in.

In server-side discovery, the client just sends to one stable endpoint — a load balancer or a virtual IP — and that does the lookup and the balancing. The client stays dumb; it doesn’t even know discovery happened. This is the Kubernetes model: a Service gives you a stable name and ClusterIP, and the platform (via kube-proxy) forwards each connection to a live pod behind it. The trade is the familiar one — a little indirection and an extra hop, bought in exchange for clients that need to know nothing about discovery. Most modern platforms choose server-side for exactly that reason.

The architect’s lens

Step back from register/health/lookup and weigh service discovery as the keystone it is:

• Why does it exist? Because instances now appear, die, and get rescheduled onto different hosts every few minutes, each with a new IP — so the old trick of hard-coding the orders box’s address is a guaranteed future outage. Discovery makes “where is service X?” a dynamic question.
• What problem does it solve? It replaces the hand-maintained map — the config files of IPs, the load-balancer member lists someone edited after every deploy — with three automatic moving parts: register, health, lookup. Health-checking is what makes a lookup return the healthy set, not merely the registered set, so the system never hands a client a corpse.
• What are the trade-offs? DNS-based discovery works in every language with no client library but suffers cache/TTL staleness (so it uses very short TTLs); registry-based (Consul, etcd, Eureka) pushes updates instantly with richer metadata but adds moving parts and a client/sidecar. Client-side discovery saves a hop but bakes logic into every client; server-side keeps clients dumb at the cost of indirection — Kubernetes chooses server-side.
• When should I avoid it? In a truly static datacenter where the set of instances never changes, a fixed IP is simpler. The machinery only earns its keep once topology changes continuously.
• What breaks if I remove it? Load balancers lose their backend list; autoscaling, self-healing, and zero-downtime deploys all collapse, because each one assumes new instances auto-register and dead ones silently drop out. It’s the bridge from “servers you tend” to “infrastructure that tends itself” — and all of Kubernetes assumes it.

Check your understanding

1. Why does hard-coding a service’s IP address work fine in a static datacenter but guarantee outages in a container environment? Which fallacy is at play?
2. Name the three moving parts every service-discovery system has, and explain what each one does.
3. Contrast DNS-based and registry-based discovery. Give one advantage of each and one drawback.
4. Why are health checks essential to discovery and not just a nice-to-have? What would lookups return without them?
5. Pick two of {autoscaling, self-healing, zero-downtime deploys} and explain how each one depends on service discovery underneath.

Show answers

1. In a static datacenter the service had a fixed IP that nothing moved, so hard-coding it worked. In a container environment instances are created, destroyed, and rescheduled constantly, each getting a new IP — so any hard-coded address is a guaranteed future outage. The fallacy is “topology doesn’t change,” violated continuously at container scale.
2. Register (an instance announces itself when it starts), health (the system keeps checking it’s alive and drops dead ones), and lookup (a client asks “where is service X?” and gets the current healthy set).
3. DNS-based discovery reuses names every language already resolves (no special client library) but suffers from caching/TTL staleness. Registry-based discovery (Consul, etcd, Eureka) pushes updates instantly and holds richer metadata, but adds moving parts and a client/sidecar that knows how to query it.
4. Without health checks, lookups would return dead instances — clients would be handed corpses and fail. Health checks are what make discovery return the healthy set, not merely the registered set, so the system routes around failure automatically.
5. Autoscaling works because new instances auto-register and start receiving traffic with no manual wiring; self-healing works because a crashed instance silently drops out of discovery and its replacement drops in — both rely on discovery keeping the live set current without a human editing a list.