Pipelines & Stages

We keep saying “the pipeline” as if it were one thing. It isn’t — it’s a structure, and once you see the structure you can read any CI/CD configuration, in any tool, without being intimidated by the YAML. A pipeline is an ordered sequence of stages, with gates between them, that a change must pass to reach production. That’s the entire concept.

Stages and gates

A stage is a logical phase of the pipeline that does one kind of work and either succeeds or fails. A gate is the rule between stages: only proceed if the previous stage passed. A failed stage closes the gate, and the change stops there — it never reaches the stages downstream, which is exactly the safety property we want.

   commit
     │
   ┌─▼────┐  gate ┌──────┐ gate ┌──────┐ gate ┌────────┐ gate ┌────────┐
   │BUILD ├──────►│ TEST ├─────►│ SCAN ├─────►│PACKAGE ├─────►│ DEPLOY │
   └──────┘  pass └──────┘ pass └──────┘ pass └────────┘ pass └────────┘
     fail        fail        fail         fail            fail
      └────────────┴────────────┴────────────┴── stop the line ──► never ships

The canonical ordering, and the reason for it:

• Build — compile the code / build the image from a clean checkout. First, because nothing downstream can run if it doesn’t build.
• Test — run unit and integration tests. Fail fast here, before spending effort on anything heavier.
• Scan — check for known-vulnerable dependencies and bad image layers (more in Image & Dependency Scanning). A security gate, not just a quality one.
• Package — produce the immutable, versioned artifact. Only after the change has earned it.
• Deploy — roll the artifact out using a strategy.

Order matters because it’s about failing as cheaply as possible. Put the fast, common failures (compile errors, unit tests) early so a bad change is rejected in seconds, not after a ten-minute build and a push to a registry. This is the “shift left” instinct: catch problems as early in the pipeline as you can, where they’re cheapest to fix.

Some gates are human

Not every gate is automated. Between staging and production you might insert a manual approval gate — a person clicks “approve.” That’s the line between Continuous Delivery and Continuous Deployment: remove the human gate and the same pipeline becomes fully automatic.

A real (small) pipeline

Here is a compact GitHub Actions pipeline showing stages as jobs, gates as needs:, and a deploy job that only runs on the main branch:

name: pipeline
on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'          # cache deps between runs
      - run: npm ci
      - run: npm test

  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm audit --audit-level=high   # fail on known high-severity CVEs

  package:
    needs: [test, scan]          # gate: only if BOTH passed
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t ghcr.io/acme/app:${{ github.sha }} .
      - run: docker push ghcr.io/acme/app:${{ github.sha }}

  deploy:
    needs: [package]             # gate: only after a successful package
    if: github.ref == 'refs/heads/main'   # only deploy from main
    runs-on: ubuntu-latest
    steps:
      - run: echo "roll out ghcr.io/acme/app:${{ github.sha }}"

Read it through the structure: test and scan are independent so they run in parallel; package declares needs: [test, scan], which is the gate — it won’t start until both succeed; deploy needs package and gates on the branch with if:. The image is tagged with github.sha, the commit hash, so the artifact is traceable back to the exact source it came from — a thread we pick up in the next page.

Two things that make pipelines fast: caching and parallelism

A slow pipeline is a pipeline people route around. If CI takes forty minutes, developers stop waiting for it, batch their changes, and you’ve quietly lost the fast feedback loop that was the whole point. So speed is not a luxury — it’s what keeps the discipline alive. Two levers:

Caching avoids redoing identical work. Downloading and installing dependencies is often the slowest step, and it rarely changes between runs. A cache keyed on the lockfile restores node_modules (or the Go/Maven/pip cache, or Docker layers) from a previous run instead of fetching everything again. In the example, cache: 'npm' does exactly this. The rule: cache things that are expensive to produce and deterministic from a key (usually a hash of the lockfile or Dockerfile).

Parallelism runs independent stages at the same time. In the YAML, test and scan have no dependency on each other, so the runner executes them simultaneously — total time is the longer of the two, not their sum. The same applies to splitting a big test suite across several machines (“sharding”). Anything with no data dependency can run concurrently.

   SERIAL (slow)              PARALLEL (fast)
   test ─► scan ─► package    test ─┐
    4m  +  2m  =  6m+         scan ─┴─► package
                               max(4m,2m) = 4m+

This is the book’s thread at the level of the assembly line itself. A pipeline removes the manual judgment of which steps to run, in what order, and whether the last one passed before doing the next — it encodes the order and the gates so a change can only advance by earning each step. The safety is structural: a vulnerable dependency caught in scan never gets packaged; an artifact that never gets packaged never gets deployed. The gate doesn’t rely on anyone remembering to check — it is the check, and it’s the same every single time.

By the numbers — a flaky gate is a gate nobody trusts

A gate only works if red reliably means “broken.” Flaky tests — ones that pass or fail nondeterministically on identical code — quietly poison that signal. Google, reporting on its own enormous test corpus in the mid-2010s, found that roughly 1 in 6 of its tests showed some flakiness:

flaky tests ≈ 16% of the suite
team's rational response → "just re-run CI" on every red
net effect → real failures get re-run away too

Once people learn to re-run a red build instead of trusting it, the gate has already failed — a genuine break can slip through on a lucky green. That’s why mature pipelines quarantine and fix flaky tests aggressively: a gate’s whole value is its trustworthiness, and flakiness spends that down.

The architect’s lens

Five questions for the pipeline structure itself:

• Why does it exist? To encode both the order of work and the rule between steps — only proceed if the previous stage passed — so a change can advance only by earning each gate, the same way every time.
• What problem does it solve? Manual judgment about which steps to run and whether the last one passed: a closed gate guarantees a bad change (a failed scan, a failed test) never reaches the stages downstream, so it can’t be packaged or deployed.
• What are the trade-offs? Order is chosen to fail cheap (build → test → scan → package → deploy), and speed must be actively defended with caching and parallelism — a 40-minute pipeline gets routed around, and flaky tests (Google: ~1 in 6) train people to re-run red until the gate is worthless.
• When should I avoid it? You rarely avoid the structure; you keep it lean — over-staging a trivial project adds latency without buying safety.
• What breaks if I remove it? Without gates, a vulnerable dependency or a failing test can flow straight to package and deploy, because nothing structurally stops it — the check stops being the check.

Check your understanding

1. Define “stage” and “gate” in your own words. What does a closed gate guarantee about the stages downstream of it?
2. Why is build-then-test-then-scan-then-package-then-deploy the canonical order? What principle drives it?
3. In the example, package has needs: [test, scan] but test and scan have no needs. What does that tell you about how those three jobs are scheduled?
4. What kinds of work are good candidates for caching, and what property must a cache key have to be safe?
5. A manual approval between staging and production is “just another gate.” How does adding or removing it change which term from the previous page applies to your pipeline?

Show answers

1. A stage is a logical phase that does one kind of work and either passes or fails; a gate is the rule “only proceed if the previous stage passed.” A closed gate guarantees the change never reaches any stage downstream of it — a bad change is stopped before it can be packaged or deployed.
2. The order fails as cheaply as possible: put the fast, common failures (compile errors, unit tests) first so a bad change is rejected in seconds rather than after a long build and a registry push. The driving principle is shift left — catch problems as early as possible, where they’re cheapest to fix.
3. Because test and scan have no needs, they have no dependency on each other and run in parallel; package declares needs: [test, scan], so it acts as a gate and won’t start until both have succeeded.
4. Cache work that is expensive to produce and deterministic from a key — dependency installs, language caches, Docker layers. The key must be derived from the inputs (a hash of the lockfile or Dockerfile) so a stale cache is never reused after those inputs change.
5. With the manual approval present, a human decides when prod gets the change — that’s Continuous Delivery. Remove the gate and the same pipeline ships to prod automatically on every pass — that’s Continuous Deployment. The term that applies hinges on exactly that one gate.