State & Drift

The Terraform page ended on a question: how does plan know what already exists? The answer is the most important — and most misunderstood — concept in Terraform: state. Get state wrong and you get duplicated resources, corrupted infrastructure, and teams overwriting each other. Get it right and Terraform’s whole safety story works.

Why state has to exist

Terraform’s job is to make reality match your code. To compute the difference between the two, it needs to know three things at every plan:

   ┌──────────────┐     ┌──────────────┐     ┌──────────────┐
   │ DESIRED      │     │ LAST-KNOWN   │     │ ACTUAL       │
   │ your .tf code│     │ the state    │     │ the real     │
   │ (the what)   │     │ file (memory)│     │ cloud (truth)│
   └──────┬───────┘     └──────┬───────┘     └──────┬───────┘
          └─── plan compares all three to compute the diff ───┘

Your code says what you want. The cloud holds what is. But Terraform also needs a record of what it created and the identities it assigned — the AWS instance ID i-0abc123, for example, which appears nowhere in your HCL. The state file (terraform.tfstate, JSON) is that record: a map from each resource in your code (aws_instance.web) to the real-world object it manages (i-0abc123), plus the attributes it last saw.

Without state, Terraform couldn’t tell the difference between “this resource doesn’t exist yet, create it” and “this resource exists, I made it, leave it alone.” Every apply would try to create everything from scratch. State is Terraform’s memory of what it owns.

State can contain secrets — treat it as sensitive

Resource attributes get written to state, and some are sensitive: database passwords, generated keys, private IPs. The state file is not an artifact to commit to a public repo or email around. This is one reason remote state (below) matters — it can be encrypted and access-controlled.

Remote state and locking

By default state is a local file. That’s fine for one person learning, and a disaster for a team. If state lives on your laptop, your teammate can’t see what’s deployed, and two people running apply at the same time will fight over the same resources — a classic race condition that can corrupt infrastructure.

The fix is a remote backend: store state in a shared, durable location (an S3 bucket, HCP Terraform, a GCS bucket) instead of locally.

terraform {
  backend "s3" {
    bucket         = "acme-terraform-state"
    key            = "prod/network.tfstate"
    region         = "us-east-1"
    use_lockfile   = true                # ← enables locking (S3-native, Terraform ≥ 1.10)
    encrypt        = true
  }
}

Remote state buys two things:

• Shared truth — everyone (and the CI/CD pipeline) reads and writes one authoritative state, so it doesn’t matter who runs Terraform or from where.
• Locking — before mutating state, Terraform acquires a lock. A second apply waits (or fails fast) instead of running concurrently. This is the same mutual-exclusion idea that keeps two processes from corrupting a shared file — applied to your whole infrastructure. Without it, simultaneous applies are the fastest way to wreck a production environment.

   Local state                       Remote state + locking
   ─────────────────────             ─────────────────────────────────
   state on one laptop               state in a shared bucket
   teammates can't see it            everyone reads one source of truth
   two applies → corruption          lock serializes applies → safe
   CI can't run it                   CI runs Terraform like anyone else

Drift: when reality stops matching state

The state file records what Terraform last saw. But the real world can change behind Terraform’s back — someone opens a port in the console, an autoscaler resizes a group, a resource is deleted by hand. Now there are three values in play and they no longer agree:

• Code says: instance type t3.micro.
• State says: last I checked, it was t3.micro.
• Reality says: someone changed it to t3.large in the console yesterday.

That gap between your code/state and the real world is drift, first introduced in Why IaC. Drift quietly breaks the promise that your repo describes reality.

plan is the drift detector

Here is where it all comes together. terraform plan doesn’t just compare code to state — it refreshes state by reading the real world first, then diffs. So plan surfaces drift automatically:

terraform plan

#   ~ resource "aws_instance" "web" {
#       ~ instance_type = "t3.large" -> "t3.micro"   # drift caught!
#     }
#
# Plan: 0 to add, 1 to change, 0 to destroy.

The ~ line is Terraform telling you: reality drifted to t3.large, your code says t3.micro, and apply will pull it back. plan is the diff between your declared intent and the actual world, with state as the bookkeeping that makes the comparison possible. Run it in CI on a schedule and you have a standing drift detector that catches every out-of-band change before it becomes a mystery.

The thread for this page

The manual, error-prone step state-with-plan removes is knowing what’s actually deployed and how it differs from intent. Without it you’d reverse-engineer the live console by eye and hope you caught everything. State gives Terraform a precise memory; plan turns that memory into an exact, reviewable diff against reality. Production is safer because every divergence — yours or a teammate’s 2 a.m. console fix — becomes visible and correctable instead of a silent landmine.

The deeper cure for drift isn’t catching it faster — it’s building infrastructure that can’t drift in the first place. That’s the idea behind immutable infrastructure. But first, a different layer of the problem: once a server exists, what configures the software inside it? That’s configuration management.

Under the hood — what the lock actually is, and the force-unlock footgun

The “lock” in remote state isn’t magic — it’s a tiny record the backend writes before mutating state and deletes after. With the S3 backend it’s a small lock file written to the bucket itself via a conditional write (older setups used a DynamoDB table item instead — deprecated since Terraform 1.11); with HCP Terraform it’s a server-side flag. While that record exists, a second apply refuses to start (“Error acquiring the state lock”) and waits or fails fast. The footgun is terraform force-unlock: if an apply is genuinely still running somewhere and you force the lock open because it “looks stuck,” you’ve re-created the exact concurrent-write race the lock existed to prevent — two processes writing state at once, which is how state files get corrupted. The rule: a lock that won’t release is a question to investigate (whose apply holds it?), not a button to mash.

The architect’s lens

Five questions for state — and for adopting a remote backend with locking:

• Why does it exist? Because to compute a diff Terraform needs a memory of what it built — the AWS instance ID i-0abc123 appears nowhere in your HCL — so the state file maps each resource to the real object it manages.
• What problem does it solve? Without state every apply would recreate everything; with it, plan refreshes against reality and surfaces drift as an exact ~ diff, so a 2 a.m. console fix becomes visible and correctable instead of a silent landmine.
• What are the trade-offs? State can hold secrets (passwords, generated keys), so it must be encrypted and access-controlled; a team needs a remote backend with locking (S3 with use_lockfile, HCP Terraform) — and force-unlock re-creates the very concurrent-write race the lock exists to prevent.
• When should I avoid it? A local state file is fine for one person learning, and a throwaway stack doesn’t need a remote backend — but the moment two people or CI touch it, local state is a corruption risk.
• What breaks if I remove it? With no state, Terraform can’t tell “create this” from “I already made this”; with no remote backend or locking, teammates run blind and concurrent applies corrupt infrastructure.

Check your understanding

1. Terraform needs three values to compute a plan. Name them and say where each one lives.
2. The AWS instance ID i-0abc123 appears nowhere in your HCL. Why does Terraform still need to know it, and where is it stored?
3. Why is local state a problem for a team? Name the two things a remote backend with locking gives you, and what each one prevents.
4. Define drift in terms of code, state, and reality. Give a concrete example of how it gets created.
5. Explain how terraform plan detects drift. What does the ~ symbol in plan output mean, and why is running plan on a schedule useful?

Show answers

1. Desired — your .tf code (the what); last-known — the state file (Terraform’s memory of what it built); actual — the real cloud (the truth). plan compares all three to compute the diff.
2. The instance ID is the real-world identity Terraform assigned but which appears nowhere in your HCL. Terraform needs it to map your resource (aws_instance.web) to the object it manages, so it can tell “create this” from “I already made this, leave it alone.” It’s stored in the state file.
3. Local state lives on one laptop, so teammates can’t see what’s deployed and two simultaneous applys race and can corrupt infrastructure. A remote backend with locking gives shared truth (one authoritative state everyone, including CI, reads/writes — prevents divergent views) and locking (serializes applies — prevents concurrent-apply corruption).
4. Drift is when reality stops matching your code/state: code says t3.micro, state last saw t3.micro, but reality is t3.large. It gets created by an out-of-band change — e.g. someone resizes the instance in the console, an autoscaler edits a group, or a resource is deleted by hand.
5. plan first refreshes state by reading the real world, then diffs — so an out-of-band change shows up. The ~ means an in-place change: the live value differs from your code and apply will pull it back (e.g. t3.large -> t3.micro). Running plan on a schedule turns it into a standing drift detector that catches every divergence before it becomes a mystery.