Refresh, Drift, and Reconciliation

Scenario: An operator, responding to an urgent capacity issue, manually resizes a server directly in the FakeCloud console. The very next terraform plan insists on shrinking it right back down to the size written in the configuration.

Refresh is the step, normally run automatically as part of plan, where Terraform reads the live, current state of every managed object straight from the provider's API and updates its in-memory picture before computing a diff. Drift is what we call it when that refreshed reality doesn't match what state expected - meaning something changed the real object outside Terraform's own apply process, exactly as happened in the console above.

When drift shows up, there are really only three honest responses, and picking the right one is a judgment call, not something Terraform can decide for you: revert the manual change by applying the existing configuration as-is (appropriate if the console change was an unauthorized or accidental fix); adopt the manual change by editing your configuration to match the new reality (appropriate if the operator's fix genuinely reflects the new correct desired state); or explicitly ignore that one field going forward if it's meant to be owned by something other than Terraform (the ignore_changes lifecycle rule from a later module). terraform plan -refresh-only and apply -refresh-only let you update Terraform's recorded state to match the observed reality without proposing any of your configured infrastructure changes - useful for capturing what happened during an incident before you decide what to do about it - but note this only updates the record; it does not by itself change your future intent, which still lives in the .tf files.

Analogy: Your configuration is the officially approved map of a piece of land; state is yesterday's survey of what's actually built on it; refresh is sending a surveyor back out to check today. When the fresh survey disagrees with yesterday's, that's a genuine discrepancy needing a human, governance-level decision about which one is right going forward - not something a computer should silently resolve for you either way.

A worked configuration

terraform plan
terraform plan -refresh-only
terraform apply -refresh-only
terraform state show fakecloud_server.api

Frequent, recurring drift on the same field is itself a signal worth paying attention to: it usually means unclear ownership between two systems, an emergency procedure that fixes things by hand in a crisis but never feeds the fix back into version-controlled code, or a field that genuinely should be actively managed by something other than Terraform (an autoscaler, say). Build a habit of monitoring drift over time, and make sure every legitimate break-glass manual change is followed promptly by a pull request that updates the configuration to match - otherwise the same drift will keep resurfacing on every future plan.

Common mistakes in production

• Treating apply -refresh-only as if it "fixes" drift permanently - it only updates Terraform's recorded state to match observed reality; it doesn't change the .tf configuration, so a normal apply afterward can still try to revert the same change.
• Silently absorbing repeated drift on the same field into muscle memory ("oh yeah, just re-apply, that always happens") instead of investigating and fixing the underlying ownership conflict.

Goal: Treat the plan as a change contract: understand every create, update, replacement, and destroy before approving it. Reinforce this lesson in the tf-state-drift-recovery Terraform lab. Open /labs/terraform and choose slug tf-state-drift-recovery; the lab runs real Terraform against the offline FakeCloud provider.