The Network Stack (TCP/IP)

A network has one job: move bytes from a process on one machine to a process on another. That sounds simple until you list everything that can go wrong — the bytes can be lost, reordered, duplicated, corrupted, or sent faster than the receiver can handle, across hardware nobody controls. The genius of TCP/IP is that it solves these problems in layers, each one ignorant of the others, so that no single piece has to understand the whole journey.

Why layers

Imagine sending a letter. You write it (the content). You put it in an envelope with an address (where it goes). The post office routes it hop by hop (how it travels). The truck carries it (the physical move). Each role is independent: you don’t care which trucks are used; the truck driver doesn’t read your letter. Networking works the same way, and the payoff is huge — you can swap Wi-Fi for Ethernet without changing your app, because the layers don’t depend on each other’s internals.

   LAYER            JOB                          EXAMPLE
   Application       what the bytes mean          HTTP, DNS, SSH
   Transport        deliver to the right process  TCP, UDP   (ports)
   Internet         route across networks         IP         (IP addresses)
   Link             move bits on one hop          Ethernet, Wi-Fi  (MAC)

The textbook OSI model has seven layers; the practical TCP/IP model has these four, and they’re what you actually debug. Read top-down it’s “meaning → process → machine → wire.” Each layer adds its own header to your data and hands the bundle down; the receiver strips them back off on the way up. That nesting is called encapsulation.

IP addresses and subnets

The Internet layer gives every machine an address. An IPv4 address is four bytes, written as 203.0.113.10. IPv6, the newer scheme, is much larger (2001:db8:220:1:248:1893:25c8:1946) and exists because the world ran out of IPv4 addresses — there are only ~4.3 billion, fewer than there are phones.

A subnet is a contiguous block of addresses that share a prefix, written in CIDR notation like 10.0.1.0/24. The /24 means “the first 24 bits are the network, the last 8 identify hosts” — so this subnet holds 10.0.1.0 through 10.0.1.255 (256 addresses). Subnets matter to DevOps because cloud networks (VPCs) are built from them: you carve a private network into subnets to separate, say, public-facing load balancers from private databases.

Some address ranges are private — reserved for internal networks and never routed on the public internet:

   10.0.0.0/8        10.x.x.x          (16 million addresses)
   172.16.0.0/12     172.16-31.x.x
   192.168.0.0/16    192.168.x.x       (your home router lives here)
   127.0.0.0/8       loopback — 127.0.0.1 is "this machine, myself"

localhost is real networking

127.0.0.1 (a.k.a. localhost) is a real IP address that loops straight back to your own machine. When you run a dev server “on localhost,” you’re using the full TCP/IP stack — it just never leaves the box. This is why everything you learn here works on your laptop.

TCP vs UDP: two delivery contracts

The Transport layer sits above IP and adds the idea of a port — a number that says which process on the machine should get these bytes (your web server on 443, your database on 5432). We cover ports in depth in Ports & Firewalls. There are two main transport protocols, and they make opposite promises.

	TCP	UDP
Promise	reliable, ordered, no duplicates	best-effort, fire-and-forget
Connection	yes (handshake first)	no
Overhead	higher	minimal
Use cases	HTTP, SSH, databases	DNS, video, gaming, metrics

TCP is a connection with guarantees: it acknowledges every chunk, retransmits anything lost, and reassembles data in order. UDP just throws a packet and hopes — which is exactly right when a dropped packet is cheaper to skip than to resend (one stale frame of video doesn’t matter; pausing to resend it does). Most of what a DevOps engineer touches is TCP.

The three-way handshake

TCP can’t just start sending — both sides must first agree to talk and synchronize their counters. That agreement is the three-way handshake:

   Client                         Server
     |  ---- SYN  (let's talk) ---->  |
     |  <--- SYN-ACK (ok, you too?) - |
     |  ---- ACK  (confirmed) ------>  |
     |                                 |
     |  === connection established === |

Three messages, one full round trip before any real data flows — the client’s first data can ride right behind its final ACK. This is why opening many short connections is slow, and why HTTP keep-alive (reusing one connection for many requests) is such a win — you pay for the handshake once. It’s also why latency compounds: a handshake across the planet costs a full round trip before the request even begins.

When you see SYN floods in security contexts, or a RST (reset) tearing a connection down, or connections stuck in TIME_WAIT, these are the handshake’s states leaking into your logs.

Packets and MTU

Data doesn’t cross the wire as one stream — it’s chopped into packets (the IP layer’s unit). Each link has a Maximum Transmission Unit (MTU), the largest packet it’ll carry, typically 1500 bytes on Ethernet. Anything bigger gets fragmented or dropped. MTU usually hides in the background, but it surfaces in nasty ways: VPNs and tunnels add headers that shrink the usable MTU, and a mismatch causes the classic “small requests work, large ones hang” bug — one of the most baffling outages until you know to suspect it.

The toolkit: see it for yourself

These four commands let you observe every layer above.

ping example.com          # IP+ICMP: is the host reachable, and how far (latency)?
ping -c 4 8.8.8.8         # send 4 packets to Google DNS; watch round-trip times

curl -v https://example.com   # walk DNS -> TCP handshake -> TLS -> HTTP, verbosely

ss -tlnp                  # TCP (t) Listening (l) sockets, numeric (n), with process (p)
ss -tn state established  # who am I currently connected to?

ping measures the Internet layer (round-trip time and packet loss). ss (the modern replacement for netstat) shows the Transport layer — every socket, its state, and which process owns it; it’s your first stop for “is anything even listening on that port?” curl -v narrates the whole stack, which makes it the single most useful debugging command in this entire part.

The thread

Before TCP, reliable delivery over an unreliable network was your application’s job — you’d hand-write retransmission, ordering, and acknowledgement logic, and get it subtly wrong. TCP removes that error-prone manual work entirely and does it correctly for everyone. The stack makes production safer by turning “messages might arrive, corrupted, in any order” into a clean, ordered pipe you can simply trust.

By the numbers — why a transatlantic round trip can’t beat ~56 ms

The page says latency compounds and a handshake costs a full round trip. Put real numbers on it. Light in optical fiber travels at about two-thirds of its vacuum speed:

   speed in fiber       ~ 200,000 km/s   (about 2/3 of 299,792 km/s)
   New York to London   ~ 5,600 km one way
   one-way minimum      = 5,600 / 200,000  ~ 28 ms
   round-trip minimum   = 2 x 28           ~ 56 ms      <- physics floor

Real measured RTT is more like 70–80 ms once you add routers, switches, and a fiber path that isn’t a straight line — and no optimization will ever beat the ~56 ms floor, because that’s the speed of light. Now layer the protocols on top: a TCP handshake costs a full round trip, and a TLS handshake adds one or two more before the first byte of your request. That is why HTTP keep-alive and connection reuse are such large wins — you pay those round trips once per connection, not once per request.

The architect’s lens

Step back from packets and handshakes and reason about the layered stack as a design choice:

• Why does it exist? Because moving bytes between processes can fail in a dozen ways — loss, reordering, duplication, corruption, overrun — and the genius is solving each in its own layer (meaning → process → machine → wire), so no single piece understands the whole journey.
• What problem does it solve? Coupling between layers. Encapsulation lets you swap Wi-Fi for Ethernet without touching your app, and TCP turns “messages might arrive corrupted in any order” into a clean ordered pipe — removing the retransmission/ordering logic you’d otherwise hand-write and get subtly wrong.
• What are the trade-offs? TCP’s guarantees cost a three-way handshake (a full round trip before any data) and per-chunk acknowledgement; UDP drops all of that for best-effort speed. Latency compounds and can’t beat the speed-of-light floor (~56 ms NY↔London), which is why keep-alive and connection reuse pay the handshake once per connection, not per request.
• When should I avoid TCP? When a dropped packet is cheaper to skip than to resend — live video, gaming, metrics — reach for UDP, because pausing to retransmit a stale frame is worse than losing it.
• What breaks if I ignore the lower layers? They surface as the bugs you can’t explain: a shrunk MTU behind a VPN gives the classic “small requests work, large uploads hang” failure; leaked handshake states (SYN floods, RST, TIME_WAIT) fill your logs — all invisible until you know which layer to suspect.

Check your understanding

1. Why are layers worth the complexity? Give one thing you can change at a lower layer without touching any higher layer.
2. What does a port add that an IP address alone can’t express? Why do you need both to reach a specific service?
3. Contrast TCP and UDP’s promises. Give one application where UDP’s “best effort” is the better choice and explain why.
4. Walk through the three-way handshake message by message. Using it, explain why HTTP keep-alive improves performance.
5. A user reports that small API requests succeed but large uploads hang forever. Which lower-layer concept from this page would you suspect, and why?

Show answers

1. Layers let each part change independently of the others. You can swap Wi-Fi for Ethernet (a Link-layer change) without touching your app, because higher layers don’t depend on a lower layer’s internals — only on its contract.
2. A port identifies which process on the machine should receive the bytes; an IP address only gets you to the machine. You need both because the full coordinate is IP:port — the IP routes to the host, the port routes to the service (e.g. HTTPS on 443 vs Postgres on 5432) running on it.
3. TCP promises reliable, ordered, de-duplicated delivery via a handshake and acknowledgements; UDP is best-effort fire-and-forget with no connection. UDP wins for live video or gaming, where a dropped packet is cheaper to skip than to resend — pausing to retransmit a stale frame is worse than dropping it.
4. SYN (client: let’s talk) → SYN-ACK (server: ok, you too?) → ACK (client: confirmed), then data flows. Keep-alive reuses one established connection for many requests, so you pay that handshake round trip once instead of per request.
5. MTU (Maximum Transmission Unit). Small requests fit in one packet and succeed; large uploads exceed the path’s MTU and, if a tunnel/VPN shrank it and fragmentation is broken, the big packets are silently dropped — the classic “small works, large hangs” symptom.