The Shell

The shell is the single most leveraged skill in this part. It’s the program that reads your commands and runs them — but more than that, it’s a small programming language designed for gluing other programs together. Almost every piece of automation you’ll meet later — a CI step, a Dockerfile RUN line, an Ansible task, a Kubernetes init container — is, underneath, a shell command. Shell literacy is the difference between doing a task and teaching the machine to do it for you, which is the whole game.

We’ll use bash, the default on most servers. sh is the leaner POSIX baseline; zsh is a popular interactive shell. For scripting, target bash or sh.

The Unix philosophy: small tools, composed

Linux doesn’t ship one giant program that does everything. It ships dozens of tiny programs that each do one thing well and speak a common language — lines of text. The shell’s job is to wire the output of one into the input of the next. That design is why a one-line command can answer a question that would be a small program in another language.

   ┌──────────┐  text   ┌──────────┐  text   ┌──────────┐
   │   cat    │ ──────► │   grep   │ ──────► │   sort   │ ──► your answer
   │ (read)   │  stdout │ (filter) │  stdout │ (order)  │
   └──────────┘         └──────────┘         └──────────┘

Three streams every program has

Every process gets three default channels:

• stdin (0) — input, by default the keyboard
• stdout (1) — normal output, by default the screen
• stderr (2) — error output, also the screen but a separate stream

Keeping errors on a separate stream is a small thing with big consequences: it lets you capture results while still seeing (or logging) errors, which matters enormously in automation.

Redirection: pointing streams at files

echo "hello" > out.txt      # stdout INTO out.txt (overwrites)
echo "more" >> out.txt      # stdout APPENDED to out.txt
sort < names.txt            # feed names.txt as stdin
./run.sh > app.log 2>&1     # stdout AND stderr → app.log  (2>&1 = "send stderr where stdout goes")
./run.sh 2> errors.log      # only stderr → errors.log
make 2>/dev/null            # discard errors (/dev/null is the bit-bucket)

Remember the fork/exec gap from the process model? Redirection is exactly what happens in that gap: the shell forks, rewires the child’s file descriptors, then exec’s your program. The program writes to “stdout” knowing nothing about where it actually goes. That indirection is the foundation of capturing logs without changing your code.

The pipe: the heart of it all

The pipe | connects one program’s stdout directly to the next program’s stdin — no temp file needed.

ps aux | grep nginx | grep -v grep      # find nginx processes (excluding the grep itself)
cat access.log | grep " 500 " | wc -l   # how many HTTP 500s today?
du -sh /var/log/* | sort -h | tail -5   # the 5 biggest things under /var/log

That last line is real triage you’ll use in Logs & Troubleshooting. Five small tools, zero custom code.

Exit status: how the shell makes decisions

From the process model: 0 is success, non-zero is failure, and $? holds the last command’s code. The shell turns that into control flow:

make test && ./deploy.sh         # deploy ONLY if tests pass (exit 0)
ping -c1 host || echo "down!"    # run the second part only if the first FAILED
grep -q "ERROR" app.log && alert # -q is silent; we only care about the exit code

&& (“and then, if it worked”) and || (“or else, if it failed”) are how a script reacts to the world. A CI pipeline is, at its core, a chain of && — every stage must succeed for the next to run.

Environment variables: configuration without editing code

The environment is a set of KEY=value pairs that a process inherits from its parent (another gift of fork). It’s how you pass configuration into a program without baking it into the program.

echo "$HOME" "$PATH"             # read variables
export DATABASE_URL="postgres://localhost/app"   # set one that CHILD processes inherit
NODE_ENV=production ./server     # set it just for this one command
env | sort                       # list the whole environment

export is the keyword that matters: an exported variable is passed down to child processes; a non-exported one stays in your shell. This is the standard way to configure software in containers and CI — twelve-factor config, secrets injected at runtime, all of it rides on environment variables. (We return to this in config & secrets.)

The text-wrangling quartet: grep, find, sed, awk

These four show up in nearly every real script. Learn them as a kit.

grep — search for lines matching a pattern.

grep -i "error" app.log            # case-insensitive
grep -r "TODO" ./src               # recurse through a directory
grep -c "404" access.log           # count matching lines
grep -E "warn|error" app.log       # extended regex: this OR that

find — locate files by name, age, size, type, then act on them.

find /var/log -name "*.log" -mtime +7        # .log files modified MORE than 7 days ago
find . -type f -size +100M                   # files over 100 MB
find /tmp -name "*.tmp" -delete              # find AND delete in one shot
find . -name "*.sh" -exec chmod +x {} \;     # run a command per match ({} = the file)

sed — stream editor; transform text, classically search-and-replace.

sed 's/localhost/db.internal/g' config.tmpl   # replace all (g) occurrences, print result
sed -i 's/DEBUG/INFO/g' app.conf              # -i edits the file IN PLACE
sed -n '10,20p' big.log                       # print only lines 10–20

awk — column-aware processing; great when data is in fields.

awk '{print $1}' access.log                    # first whitespace-separated field (the IP)
awk -F: '{print $1}' /etc/passwd               # -F sets the field separator to ":"
awk '$9 == 500 {print $7}' access.log          # URLs (field 7) where status (field 9) is 500

When to use which

grep finds lines, sed edits lines, awk understands columns, find locates files. If your data is “rows and columns,” reach for awk. If it’s “replace this text,” reach for sed. Don’t force one tool to do another’s job — composing them with pipes is the point.

From commands to scripts

A shell script is just commands in a file. Two lines make it robust:

#!/usr/bin/env bash
set -euo pipefail

# -e  : exit immediately if any command fails (non-zero)
# -u  : error on use of an unset variable (catches typos)
# -o pipefail : a pipeline fails if ANY stage fails, not just the last

BACKUP_DIR="${1:-/var/backups}"          # $1 = first argument; default if absent
mkdir -p "$BACKUP_DIR"
for f in /etc/app/*.conf; do             # loop over matching files
  cp "$f" "$BACKUP_DIR/" && echo "backed up $f"
done

The first line (#!/usr/bin/env bash, the shebang) tells the kernel which interpreter to run. The second line, set -euo pipefail, is the single most important habit in shell scripting.

Why set -euo pipefail is non-negotiable

By default, a shell script keeps going after a command fails. That’s the classic disaster: a backup script whose cd fails silently, so it rms files in the wrong directory; or a deploy that pushes a broken artifact because an earlier build step failed and nobody noticed. set -e makes failure stop the script instead of barrelling on. Always quote your variables ("$f") too — an unquoted variable with a space in it is how spaces in filenames turn into surprise arguments.

Why shell literacy is the core of automation

Here’s the thread for this page. The manual, error-prone step the shell removes is the human at the keyboard. Anything you can type, you can put in a script — and a script runs the same way every time, at 3 a.m., on a hundred servers, without forgetting a step or fat-fingering a flag. The leap from “I SSH in and run these commands” to “the pipeline runs this script” is the leap from artisanal operations to repeatable, reviewable, version-controlled operations. Every later tool — CI/CD, Ansible, Dockerfiles — is a more structured way of doing what a shell script does. Master the shell and the rest is vocabulary.

It actually happened — Steam’s rm -rf bug (2015)

In 2015 a bug in Valve’s Steam client for Linux turned an unset variable into a disaster. A script contained roughly:

rm -rf "$STEAMROOT/"*

Under a certain upgrade path STEAMROOT ended up empty, so the line expanded to rm -rf "/"* — recursively deleting every file the user could write, home directory and all. This is precisely the failure set -u (error on an unset variable) and careful quoting exist to prevent: with set -u the empty variable would have aborted the script instead of feeding rm a catastrophic argument. “The script kept going after something was wrong” is not hypothetical — it has eaten people’s machines. A few characters of safety boilerplate is the cheapest insurance in computing.

The architect’s lens

Step back from the syntax and ask the five questions about the shell as a tool you choose to lean on:

• Why does it exist? Because Unix ships dozens of tiny single-purpose tools that all speak lines of text, and something has to wire one program’s stdout into the next’s stdin — the shell is that “small programming language designed for gluing other programs together.”
• What problem does it solve? It removes the human at the keyboard: anything you can type becomes a script that runs the same way every time, at 3 a.m., on a hundred servers — the leap from artisanal operations to repeatable, reviewable, version-controlled ones.
• What are the trade-offs? Terseness and power come with sharp edges — an unset or unquoted variable can expand catastrophically (rm -rf "$STEAMROOT/"* becoming rm -rf "/"*), which is exactly why set -euo pipefail and quoting "$f" are non-negotiable rather than optional.
• When should I avoid it? Once a script grows real data structures, nontrivial error handling, or hundreds of lines — bash’s stringly-typed nature and quoting traps make a real language (Python, Go) the safer choice past that point.
• What breaks if I remove it? Nearly every layer above collapses — a CI step, a Dockerfile RUN line, an Ansible task, a Kubernetes init container are all shell underneath; without it you’re back to running commands by hand and hoping you don’t forget a step.

Check your understanding

1. Explain ./run.sh > out.log 2>&1. Why are stdout and stderr separate streams, and what does 2>&1 accomplish?
2. Connect redirection to fork/exec: at what moment does the shell rewire a program’s output, and why can the program stay oblivious to it?
3. Write (in words) a one-liner that counts how many lines in access.log contain 500. Which tools compose to do it?
4. What does make test && ./deploy.sh guarantee, and which property of exit codes makes it work?
5. What three failures does set -euo pipefail prevent, and why is “the script keeps going after an error” so dangerous in a deploy?

Show answers

1. It runs run.sh and sends both stdout and stderr to out.log. They’re separate streams (stdout=1 for normal output, stderr=2 for errors) so you can capture results while still seeing or logging errors independently. 2>&1 means “send stderr wherever stdout currently goes” — order matters, so the redirect to the file must come first.
2. The shell rewires output in the gap between fork and exec: it forks, redirects the child’s file descriptors (e.g. fd 1 → a file), then exec’s your program. The program just writes to “stdout” knowing nothing about where it actually points — that indirection is what lets you capture logs without changing any code.
3. grep -c "500" access.log (or grep "500" access.log | wc -l). The composing tools are grep to match the lines and wc -l to count them — or grep -c, which counts matches directly.
4. It guarantees deploy.sh runs only if make test exited 0 (success). It works because of the exit-code convention — 0 = success, non-zero = failure — and && runs its right side only when the left side succeeded.
5. -e stops the script the moment any command fails (instead of barrelling on); -u errors on an unset variable (catching typos); -o pipefail makes a pipeline fail if any stage fails, not just the last. “Keeps going after an error” is dangerous in a deploy because a failed early step (a cd that didn’t happen, a build that broke) lets later steps run against the wrong state — rm-ing the wrong directory, or shipping a broken artifact nobody noticed.