Retrieve, Augment, Generate

The three RAG steps

Retrieval-augmented generation (RAG) is the name for connecting the search system from the previous two modules to a language model, and it breaks down cleanly into the three steps its name describes: retrieve the most relevant chunks for the user's question (using the embedding-and-similarity approach covered earlier), augment the prompt by inserting those retrieved chunks into it as reference material, and then generate an answer by asking the LLM to respond using that supplied context. It's worth being precise about what this technique does and doesn't do: it supplies the model with temporary reference material for one single request. It does not retrain the model or change its underlying weights in any way - the next time you ask a question, you're back to a model that has no permanent memory of anything you just showed it.

SYSTEM: Answer only using the CONTEXT below. If the answer isn't
        in the context, say you don't know.
CONTEXT:
  [doc-runbook-14] "Restart the service with: systemctl restart api"
USER: How do I restart the API service?

MODEL: Run `systemctl restart api`, per doc-runbook-14.

That prompt structure matters for a security reason, not just an accuracy one: put clear markers around the retrieved context, and explicitly instruct the model to answer only from it. Every retrieved document should be treated as untrusted data, not as trusted instructions - because a document, especially one pulled from a source you don't fully control, might contain prompt injection: text specifically written to trick the model into ignoring the application's real instructions and doing something else instead. (This gets a full dedicated module later in the security phase of this track.) That injected text must never be allowed to override the system's real rules, no matter how convincingly it's phrased.

Warning: RAG substantially reduces unsupported, made-up answers compared to an LLM working from memory alone - but it does not guarantee correctness. The model can still misread or ignore the supplied context. Always preserve citations back to the specific retrieved chunks, so a user (or a debugging engineer) can inspect the actual evidence an answer was supposedly based on.