Where Guardrails Fit

Validate before and after generation

A guardrail is any control that constrains what an AI application is willing to accept as input, willing to actually do, or willing to return as output - a deliberately broad definition that covers a lot of what this security phase of the track has already touched on individually. It's useful to think of guardrails as belonging to three checkpoints in the request's lifecycle. Input guardrails run before the expensive step of actually calling the model, filtering out what shouldn't even be attempted. Tool guardrails run before any action is actually taken, echoing the allowlist principle from the terminal-agents module earlier in this track. Output guardrails run after generation, before a response is allowed to reach a real user or get consumed by a downstream program.

Common input rules include a maximum character count, a list of allowed languages, detection of known blocked instruction patterns (tying back directly to the prompt-injection module), and a check that all required fields are actually present. Common output rules include requiring genuinely valid JavaScript Object Notation (JSON) rather than merely JSON-ish text, requiring specific keys to be present, restricting a status field to an explicit allowed list of values, enforcing a maximum length, and confirming the absence of anything that looks like a leaked secret (tying back to the previous module's PII masking).

Analogy: Airport security enforces different rules at several distinct doors, not just one: ticket validation happens before you even enter the terminal, additional screening happens before restricted areas, and customs happens again on the way out. No single checkpoint, however thorough, can realistically enforce every rule that matters across the whole journey - which is exactly why guardrails belong at multiple points, not just one.

Scenario: An application's documented contract promises callers a clean {answer, status} JSON shape, but its actual internal policy accidentally still allows an internal debug status value to leak through in certain code paths. Internal debugging details meant only for developers can then reach real clients, and downstream automation has no reliable way to distinguish a genuinely valid, complete response from a broken, incomplete one - because nothing was actually enforcing the promised contract in code.

The core lesson to take from that scenario: enforce these contracts deterministically, in application code, not by simply asking the model nicely. Instructing the model with something like "please return valid JSON" in the prompt is helpful guidance that improves the odds, but it is not validation - the model can still occasionally get the shape wrong, and something in the application actually has to check. Invalid output must be explicitly rejected, corrected through a narrow and bounded repair path, or returned to the caller as a clearly labeled controlled error - never silently passed through and hoped to be fine.