Package Management

You need software on your servers — a web server, a language runtime, a database client. The naive way is to download a tarball, compile it, and copy files around by hand. The Linux way is a package manager: a tool that installs, upgrades, and removes software while tracking what depends on what. This page is where the book’s central question gets sharp: reproducibility. A package manager is the first tool that lets you say “install exactly this” and get the same result every time — the antidote to “but it worked on my machine.”

What a package actually is

A package is an archive bundling a program’s files plus metadata: its version, a list of other packages it needs (dependencies), and scripts to run on install/remove. The two dominant formats split the Linux world:

• .deb — Debian, Ubuntu, and their derivatives. Tool: apt (low-level: dpkg).
• .rpm — RHEL, Fedora, Rocky, Amazon Linux. Tool: dnf (older: yum; low-level: rpm).

The concepts are identical; only the commands differ. Here they are side by side:

Task	Debian/Ubuntu (apt)	RHEL/Fedora (dnf)
Refresh package lists	sudo apt update	(automatic)
Install a package	sudo apt install nginx	sudo dnf install nginx
Upgrade everything	sudo apt upgrade	sudo dnf upgrade
Remove a package	sudo apt remove nginx	sudo dnf remove nginx
Search	apt search nginx	dnf search nginx
Show details	apt show nginx	dnf info nginx
List installed	apt list --installed	dnf list installed
What owns this file?	dpkg -S /usr/sbin/nginx	rpm -qf /usr/sbin/nginx

update ≠ upgrade on Debian

A trap worth internalizing: on apt, update refreshes the list of available packages (it installs nothing), while upgrade actually installs newer versions. You almost always run sudo apt update before installing, so apt knows what’s currently available. On dnf, the refresh is handled for you.

Dependencies, and the problem they solve

Software is built on other software. nginx needs an SSL library; that library needs a C library; and so on. Resolving this by hand is miserable and error-prone — the infamous “dependency hell” where installing A breaks B because they need different versions of C.

The package manager solves it by reading the metadata and computing the whole set of packages required, then installing them together:

   you ask for:           nginx
   manager resolves:      nginx
                          ├── libssl3        (TLS)
                          ├── libpcre3       (regex)
                          └── libc6          (the C standard library)
   → installs all of them, in dependency order, automatically

This is the manual step a package manager removes: figuring out and fetching every transitive dependency yourself. What used to be an afternoon of compiling is now one command — and crucially, one repeatable command.

Repositories: where packages come from

A package manager doesn’t download from random URLs. It pulls from configured repositories — curated servers hosting packages and an index of what’s available. Your distribution ships with official repos (defined in /etc/apt/sources.list and /etc/apt/sources.list.d/ on Debian; /etc/yum.repos.d/ on RHEL). You can add third-party repos for software not in the defaults (Docker, a database vendor, a language toolchain).

# Debian/Ubuntu: see what repos are configured (newer releases use deb822 ".sources" files)
grep -rE '^(deb |URIs:)' /etc/apt/sources.list /etc/apt/sources.list.d/ 2>/dev/null

# RHEL/Fedora: list enabled repositories
dnf repolist

Repos are also where trust enters the picture. Packages are cryptographically signed, and your package manager verifies the signature against the repo’s public key before installing. That signature check is what stops an attacker who tampers with a mirror from slipping you a malicious package — your first taste of supply-chain security, which becomes a major theme later in the book.

Adding a third-party repo is a trust decision

When you curl ... | sudo bash an install script or add a vendor’s repo and key, you are granting that party the ability to install software as root on every future apt upgrade. Add repos deliberately, prefer the vendor’s documented signed-repo instructions over piping a script into a shell, and know which repos you’ve trusted.

Why reproducible installs matter — and version pinning

Here’s the heart of the page. By default, apt install nginx gives you whatever version the repo offers today. Run it on Monday and on Friday and you might get different versions — and now your staging box and your production box disagree, or two “identical” servers behave differently. That drift is exactly the “works on my machine” disease, and it’s the manual-step thread of this page: an unpinned install is a quietly non-deterministic step hiding inside something that looks deterministic.

The fix is pinning — specifying the exact version, so the install is reproducible:

# Debian/Ubuntu: see available versions, then install a specific one
apt list -a nginx
sudo apt install nginx=1.24.0-1ubuntu1

# Hold it so a routine `apt upgrade` won't move it
sudo apt-mark hold nginx

# RHEL/Fedora: install an exact version
dnf list nginx --showduplicates
sudo dnf install nginx-1.24.0

# Lock it against upgrades (needs the versionlock plugin)
sudo dnf versionlock add nginx

Pinning makes a deploy deterministic: the same version, every machine, every time. It also makes upgrades deliberate — you bump the pinned version on purpose, test it, and roll it out, rather than having a background upgrade silently change your runtime under you. That deliberateness is how you make production safer: surprises become decisions.

   UNPINNED                          PINNED
   apt install nginx                 apt install nginx=1.24.0-1ubuntu1
   → Monday:  1.24.0                 → always 1.24.0
   → Friday:  1.26.1  ⚠ drift!       → upgrade is an explicit, reviewed change

Where this is heading

Package managers give you reproducible installs on a machine. But the machine itself still drifts — files change, someone hotfixes a config, two servers slowly diverge. The next steps in the book push reproducibility further: containers capture the entire filesystem as a versioned, immutable image (a Dockerfile is largely a list of pinned apt install lines), and infrastructure as code captures the servers themselves. Each layer is the same instinct — make installs declarative and repeatable — applied at a bigger scope. Package management is where that instinct begins.

It actually happened — the xz-utils backdoor (CVE-2024-3094, 2024)

In March 2024 the world got the supply-chain scare this page warns about. A malicious maintainer had spent months gaining trust on xz-utils, a tiny, ubiquitous compression library, then slipped a backdoor into the release tarballs (not the visible Git source) of versions 5.6.0 and 5.6.1. Through the dependency chain it hooked into sshd, aiming to let the attacker bypass authentication on any server running the poisoned build. It was caught almost by luck: a Microsoft engineer, Andres Freund, noticed sshd logins were about half a second slower than expected and pulled the thread. Because it was spotted before the new version reached most stable distributions, the damage was limited — but it had already landed in bleeding-edge Debian and Fedora. The lesson lands exactly where the page does: every package you install is a trust decision, and a signature only proves who built it, not that what they built is safe. (More in supply-chain security.)

The architect’s lens

A package manager is the first reproducibility tool you adopt, so hold it up to the five questions:

• Why does it exist? Because installing software by downloading tarballs and compiling by hand is miserable and non-reproducible; a package manager installs, upgrades, and removes software while tracking what depends on what.
• What problem does it solve? It reads metadata and computes the whole transitive dependency set, installing it in order automatically — ending “dependency hell” and letting you say “install exactly this,” the first real antidote to works on my machine.
• What are the trade-offs? Convenience rides on trust: every repo you add and every curl … | sudo bash grants a third party the ability to install software as root on future upgrades, and a signature proves only who built a package, not that it’s safe (the xz-utils backdoor, CVE-2024-3094).
• When is it not enough? An unpinned apt install nginx is quietly non-deterministic — Monday’s 1.24.0 versus Friday’s 1.26.1 — so pin the exact version (nginx=1.24.0-1ubuntu1, then apt-mark hold) whenever a deploy must be reproducible.
• What breaks if I skip pinning? Two “identical” servers drift apart, a routine background apt upgrade silently changes your runtime, and you lose the per-machine reproducibility that containers and infrastructure as code build on top of.

Check your understanding

1. What metadata does a package carry beyond the program’s files, and why is the dependency list the part that saves you the most manual work?
2. On Debian, what is the difference between apt update and apt upgrade, and why do you usually run update first?
3. What is a repository, and what role does package signing play? Connect it to supply-chain trust.
4. Explain how an unpinned apt install can produce drift between two servers, and how version pinning makes a deploy reproducible.
5. A package manager makes installs reproducible on one machine. Name the two later approaches in this book that extend reproducibility further, and what bigger scope each one captures.

Show answers

1. A package carries its version, a list of dependencies (other packages it needs), and install/remove scripts — not just the program’s files. The dependency list saves the most manual work because the manager reads it and computes the whole transitive set, then installs everything in dependency order automatically — sparing you “dependency hell,” where installing A breaks B over a shared library version.
2. apt update refreshes the list of available packages (it installs nothing); apt upgrade actually installs newer versions. You run update first so apt knows what’s currently available before you install — otherwise it works from a stale index. (On dnf, the refresh is automatic.)
3. A repository is a curated, configured server hosting packages plus an index of what’s available (the manager pulls only from these, not random URLs). Packages are cryptographically signed, and the manager verifies each signature against the repo’s public key before installing — so an attacker who tampers with a mirror can’t slip you a malicious package. That’s your first taste of supply-chain security.
4. By default apt install nginx gives whatever version the repo offers today, so installing on Monday vs Friday can yield different versions — two “identical” servers quietly diverge (“works on my machine” drift). Pinning the exact version (nginx=1.24.0-1ubuntu1, plus apt-mark hold) makes the install deterministic — the same version on every machine, every time — and turns upgrades into deliberate, reviewed changes.
5. (1) Containers — capture the entire filesystem as a versioned, immutable image (a Dockerfile is largely a list of pinned apt install lines). (2) Infrastructure as code — capture the servers themselves declaratively. Each applies the “make installs declarative and repeatable” instinct at a bigger scope.