The Drift Decision Tree

Scenario: During a live security incident, an engineer manually closes an exposed public port directly through the provider's console to stop the bleeding right now - and the very next Terraform plan proposes reopening it, because the configuration still says it should be open.

You met the mechanics of drift in Module 5; this section is about the judgment call every drift finding actually requires, because Terraform itself has no way to know which side - the configuration or the live emergency fix - represents the correct desired state. For every drift item, walk through the same four questions: Was the external change actually authorized, or is it itself the problem? Should the configuration be updated to adopt this change as the new correct desired state going forward? Should Terraform instead revert it, treating the external change as unauthorized or unwanted? Or does this reveal that the field's ownership genuinely needs to move to something other than Terraform (an ignore_changes case from Module 7)?

In the incident scenario above, the honest answer is almost certainly "adopt": the emergency port closure was probably the correct new desired state, and the configuration was wrong even before the incident, not the other way around - so update the .tf file to reflect the closed port, then plan and apply, rather than letting an unmodified plan silently reopen the very thing that was just closed for a security reason. plan -refresh-only (from Module 5) is useful here for capturing and recording what happened during the incident, but be precise about its limits: it updates Terraform's stored record of current reality, it does not by itself change your future intent - that still requires an actual edit to the .tf file.

Analogy: Drift is a disagreement between the official policy handbook and what's actually true on the shop floor right now. Sometimes the shop floor made an unauthorized shortcut and the handbook is right - revert it. Sometimes an emergency fix on the shop floor was the genuinely correct call, and it's the handbook that's now out of date - update the handbook, don't undo the fix.

A worked configuration

terraform plan -refresh-only
# decide and update HCL
terraform plan
terraform apply

During an active incident specifically, the order of operations matters: restore safety first (leave the manual fix in place), then go back and reconcile code, state, and documentation once the immediate danger has passed - don't let "we need to update the Terraform config properly" delay an urgent safety fix. Whatever you do, record who changed what, when, and why, in a place other than just your memory. Never let an automated process blindly "revert drift" without a human first understanding the security or operational context behind it - an automated revert here would have literally recreated the vulnerability the engineer just fixed.

Common mistakes in production

• Applying an unmodified configuration during or right after a security incident without first checking whether the drift it would revert was actually an intentional, correct emergency fix.
• Treating refresh-only as if it updates your future desired state - it only records current observed reality; the .tf configuration still needs an explicit edit to change what future plans will do.

Goal: Treat the plan as a change contract: understand every create, update, replacement, and destroy before approving it. Reinforce this lesson in the tf-state-drift-recovery Terraform lab. Open /labs/terraform and choose slug tf-state-drift-recovery; the lab runs real Terraform against the offline FakeCloud provider.