GitOps

The deployment strategies page answered how a new version replaces the old. This page answers a question we slid past: who pushes the deploy, where do they push it from, and how does anyone know what’s actually running right now? GitOps gives a deceptively simple answer — git is the single source of truth for the desired state of your system, and an automated agent continuously makes reality match git. It is the reconciliation loop from Part 4, extended to cover your whole deployment.

The problem with push-based deploys

Re-examine the pipelines so far. The deploy stage pushes changes outward: it holds cluster credentials, runs kubectl apply or helm upgrade, and reaches into the production cluster from the CI runner. This works, but it has uncomfortable properties once you look closely:

• The CI system holds prod credentials. Your build server can write to your production cluster, so it’s now a juicy target — compromise CI and you own prod.
• Drift is invisible. After the push, nothing keeps checking. If someone runs a manual kubectl edit at 2 a.m. during an incident, the cluster silently diverges from what the pipeline last applied, and nobody finds out until the next deploy mysteriously behaves differently.
• “What’s running?” has no authoritative answer. The true state lives in the cluster’s memory, not in any file you can read, review, or diff. To know what’s deployed you have to go ask the cluster — and hope no one changed it by hand.

   PUSH-BASED                         GitOps (PULL-BASED)
   ──────────                         ───────────────────
   CI ──(has prod creds)──► cluster   git repo (desired state)
        kubectl apply                       ▲   │
   then... nobody is watching               │   │ agent pulls
   drift goes unnoticed                      │   ▼
                                       agent in cluster ──► reconcile ──► cluster
                                       (loops forever, corrects drift)

The GitOps inversion

GitOps flips the direction. Instead of CI pushing into the cluster, an agent inside the cluster pulls from git. The four principles:

1. Declarative. The entire desired state of the system is expressed as declarative config — Kubernetes manifests, Helm charts, Kustomize. You describe what should exist, not the steps to create it. (This is the declarative vs imperative idea, applied to deployments.)
2. Versioned in git. That config lives in a git repository, which becomes the single source of truth. Git’s history is your deploy history: every change is a commit, with an author, a timestamp, a diff, and a code review.
3. Pulled automatically. An agent in the environment continuously pulls the desired state from git — nobody pushes credentials in from outside.
4. Continuously reconciled. The agent compares desired (git) to actual (cluster) on a loop, and corrects any difference. Reconciliation isn’t a one-shot deploy; it’s a standing process.

A deploy is a git commit

The headline consequence: to change production, you change git. Want to ship a new artifact? Open a pull request that bumps the image digest in a manifest. It gets reviewed, merged, and the agent rolls it out. The deploy is the merge. There is no separate, out-of-band “deploy action” with its own access and its own audit gap.

What this buys you

Walk back through the push-based problems and watch them dissolve.

Drift detection and self-healing. Because the agent reconciles continuously, manual changes don’t last. That 2 a.m. kubectl edit is detected as drift — the cluster no longer matches git — and the agent either flags it or reverts it. The cluster is self-healing toward the declared state. The only durable way to change production is to change git, which means every change goes through review.

Rollback is git revert. Production broke after the last merge? Revert the commit. The desired state returns to its previous, known-good version, and the agent reconciles the cluster back. You are not reconstructing a past state by hand — git preserved it, exactly, and you’re pointing back at it. This is the immutable-artifact idea extended to whole-system config.

A real audit trail. “Who changed production, when, and why?” is answered by git log. Every change is a reviewed, attributed, reversible commit. Compliance and incident forensics get a single source instead of a scavenger hunt across kubeconfigs and chat history.

Smaller credential blast radius. CI no longer needs prod cluster credentials — it only needs to push artifacts to a registry and commits to git. The agent that holds cluster access lives inside the cluster and only ever reads from git. Compromising the build server no longer hands an attacker your production cluster.

   CHANGE PROD            ROLL BACK             AUDIT
   ──────────            ─────────             ─────
   edit manifest         git revert <sha>      git log / git blame
   → PR → review         → agent reconciles     → who, when, why,
   → merge → reconcile   → known-good restored    full diff

Where the line is

GitOps describes the deploy/runtime side: the desired state and the reconciliation. It pairs with the CI pipeline, which still does the building, testing, scanning, and pushing of artifacts — CI’s last act becomes “open a PR (or commit) updating the desired-state repo” rather than “reach into the cluster.” A common split is two repos: an app repo (source code, CI) and a config repo (manifests, the GitOps source of truth).

This is the book’s thread at the level of the cluster’s whole state. The manual, error-prone steps removed are out-of-band deploys (someone with prod creds running commands), invisible drift (hand edits no one tracks), and reconstructing the past from memory during a rollback. The safety comes from a single property: the system is always being pulled back toward a reviewed, version-controlled declaration of what it should be. The cluster can’t quietly become something nobody approved. The next page makes this concrete with the most widely used GitOps agent: Argo CD.

Under the hood — GitOps reconciles config, not correctness

A subtlety worth holding onto: the agent guarantees the cluster matches git, not that what’s in git is good. Commit a syntactically valid manifest that points at a broken image and the agent will faithfully apply it — and, with self-heal on, fight you if you try to hand-fix the cluster, dragging it right back to the broken declared state. “Synced” is a statement about agreement with git, not health. That’s exactly why a GitOps tool reports health as a separate axis from sync (you’ll see Argo CD’s Synced-but-Degraded on the next page), and why GitOps pairs with tests, progressive delivery, and health gates: reconciliation makes a change reviewed and reversible, but it can’t make a bad change correct.

The architect’s lens

Five questions to decide whether GitOps belongs in your deploy path:

• Why does it exist? To fix push-based deploys, where CI holds prod credentials, drift is invisible, and “what’s running?” has no authoritative answer — by making git the desired state and an in-cluster agent pull and reconcile it.
• What problem does it solve? Out-of-band deploys and silent drift: a deploy becomes a reviewed commit, a 2 a.m. kubectl edit self-heals back to git, the audit trail is git log, and rollback is git revert.
• What are the trade-offs? It guarantees the cluster matches git, not that git is correct — a valid manifest pointing at a broken image syncs faithfully — so you still need tests, health gates, and progressive delivery, plus (usually) a two-repo split.
• When should I avoid it? When there’s no reconcilable, declarative substrate to point it at, or for genuinely imperative one-off operations a standing loop can’t express.
• What breaks if I remove it? CI needs prod cluster credentials again (a far bigger blast radius), drift goes unnoticed, and rollback returns to reconstructing a past state by hand instead of pointing back at one git never lost.

Check your understanding

1. Name two concrete problems with push-based deploys (where CI runs kubectl apply into the cluster).
2. State the four GitOps principles. Which one is responsible for the cluster “self-healing” back to the declared state?
3. Explain the phrase “a deploy is a git commit.” What does it do to your audit trail and your review process?
4. In GitOps, how do you roll back, and why is it fundamentally different from reconstructing a past state by hand?
5. Using the book’s thread, explain how pulling desired state from git (rather than pushing into the cluster) shrinks the credential blast radius and eliminates invisible drift.

Show answers

1. Any two of: CI holds prod credentials (compromise CI and you own prod); drift is invisible (a manual kubectl edit silently diverges the cluster with nothing watching); and “what’s running?” has no authoritative answer (the true state lives only in the cluster, not in any reviewable file).
2. Declarative, versioned in git, pulled automatically, and continuously reconciled. The last one — continuous reconciliation — is responsible for self-healing: the agent constantly compares git to the cluster and corrects any difference, so manual changes don’t last.
3. It means the only durable way to change prod is to change git — so every change is a reviewed, attributed, diffable commit. Your audit trail becomes git log and your review process becomes the normal pull-request gate, with no out-of-band deploy action that bypasses either.
4. You git revert the offending commit; the desired state returns to its previous, known-good version and the agent reconciles the cluster back. It’s fundamentally different because git preserved the past state exactly — you point back at it rather than reconstructing it by hand under pressure.
5. Pulling means the agent that holds cluster access lives inside the cluster and only reads from git, so CI never needs prod credentials — compromising the build server no longer hands over prod. And because the agent reconciles continuously, any hand edit is detected as drift and reverted, so the cluster can’t quietly become something nobody approved.