Match Controls to Impact

Risk comes from capability plus context

Classify use cases by potential harm, reversibility, scale, affected people, data sensitivity, and degree of autonomy. A typo in a private draft is unlike an automated account suspension. Higher-impact systems need stronger validation, least-privilege tools, human review, audit trails, fallback procedures, and sometimes should not use an LLM at all.

A human-in-the-loop control is meaningful only when the reviewer has time, context, authority, and a usable way to reject the suggestion. Automation bias is the human tendency to over-trust machine recommendations, especially when interfaces present them confidently.

low impact: draft text -> user edits
medium impact: prioritize queue -> operator can override
high impact: change access/deploy/delete -> explicit approval + deterministic checks
prohibited: action outside policy or lawful basis

Warning: A confirmation button is not sufficient oversight if reviewers habitually click through opaque suggestions. Show evidence, exact action, target, and consequences.

Note: Legal obligations differ by jurisdiction and change over time. Governance owners must obtain current qualified guidance rather than treating a curriculum as legal advice.