Retrieval before response
Retrieval-augmented generation, usually shortened to RAG, adds a search step to a language-model answer. Instead of asking the model to rely only on patterns learned during training, a system finds relevant passages from an approved collection and includes them in the model's working context. The model then writes an answer using those passages. This can make a private policy manual or changing product catalog more useful without retraining a model for every update.
The critical word is “relevant.” If search returns the wrong document, an obsolete policy or a passage from another customer's files, a polished answer may still be wrong. The retrieval system must respect permissions and document versions. Breaking files into searchable chunks helps, but chunk size and metadata can change whether the right context is found. A good implementation tracks which passages supported each answer.
Where errors appear
RAG does not eliminate hallucinations. The model may overstate a passage, blend two incompatible policies or answer a question for which no retrieved text is sufficient. Long documents can contain contradictory sections. A malicious document can try to instruct the assistant to ignore its rules. These are reasons to test both search and generation separately.
Create a small benchmark of real questions with known answers, including outdated-policy traps and questions with no answer. Measure whether the correct passage was retrieved, then whether the final answer faithfully reflects it. If retrieval fails, rewriting the prompt alone will not repair the knowledge base.
A useful design rule
For important answers, show the internal document title and last-reviewed date, and let the user open the relevant passage. Require the assistant to say when evidence is missing or conflicting. Maintain owners and review dates for the underlying documents. RAG is best understood as a pipeline for finding and using evidence, not as a truth switch. Its usefulness rises with document quality, permission discipline and the ability to inspect why an answer was given.


