RAG (retrieval-augmented generation)

Retrieval-augmented generation, or RAG, is a way of building AI systems that answer from a specific body of knowledge. When a question comes in, a retrieval step searches a document collection for the most relevant passages. Those passages are added to the prompt, and the model writes an answer based on them, ideally with citations.

RAG solves two problems at once. Models do not know your private documents, and their built-in knowledge stops at a training cutoff. Fine-tuning a model on your documents is expensive and does not reliably teach facts. Retrieval lets a general model answer questions about a handbook, product docs or a contract library that changes every week, simply by updating the documents.

A typical RAG pipeline splits documents into chunks, turns each chunk into an embedding, stores them in a vector database, and finds the closest chunks to each question. Many systems combine this with keyword search and a re-ranking step. Quality depends heavily on retrieval: if the right passage is not found, the model can not use it.

Example

An HR chatbot gets the question "How many days of parental leave do I get?". Retrieval finds two passages from the leave policy and one from a recent policy update. The model answers from those, notes that the update changed the number from 12 to 16 weeks, and cites both.

Common mistake

Blaming the model when the retrieval is at fault. If answers are wrong, first check which passages were retrieved. Poor chunking, missing documents or vague questions cause more RAG failures than the language model itself.