Skip to content

RAG -- Retrieval-Augmented Generation

  • Intermediate · Module 01
  • Lesson 5 of 7 · then sources and quiz

Step 1 of 4Knowledge

LLMs have a fundamental problem: their knowledge is limited to their training data. A model trained in January 2026 knows nothing about events after that date. It also doesn't know about internal company documents, private databases, or daily updates.

Retrieval-Augmented Generation (RAG) solves this problem. Instead of retraining the model, you provide relevant information directly in the prompt. RAG combines two strengths: the LLM's ability to generate natural language with the precision of a database.

Note
RAG in One Sentence

RAG = First find the right information, then let the LLM answer based on it.

Step 2 of 4Understand

The RAG Pipeline in Six Steps

Flow · RAG pipeline
Click a step to see its explanation. The animation shows the data flow through the RAG pipeline.
  1. Query: The user asks a question, e.g., "What is the return policy for online orders?"
  2. Embedding: The question is converted into an embedding vector -- using the same model that embedded the documents.
  3. VectorDB: The question vector is matched against the vector database holding all document embeddings.
  4. Retrieval: The top-K most similar documents are fetched and prepared as context for the LLM.
  5. LLM: The language model receives the question plus the retrieved documents and generates an answer grounded in both.
  6. Answer: The grounded response is returned -- backed by current, relevant sources rather than training data alone.

RAG vs. Pure Prompting

AspectPure PromptingRAG
Knowledge SourceTraining data onlyTraining data + external documents
FreshnessTraining cutoff dateAs current as the database
HallucinationsHigher (no fact-checking)Lower (answers grounded in sources)
CitationsNot possiblePossible (document references)
EffortMinimalEmbedding pipeline + vector DB required
Question · Self-check
Why does RAG reduce hallucinations compared to pure prompting?

Step 3 of 4Apply

When to use RAG and when not to

RAG makes sense for:

  • Company-specific knowledge (internal documentation, policies)
  • Frequently changing information (product catalogs, pricing)
  • Areas where citations matter (compliance, legal)

RAG is overkill for:

  • General knowledge that any LLM knows ("What is the capital of France?")
  • Creative tasks without a factual basis (writing poems, brainstorming)
  • Simple conversations without domain-specific content

The Indexing Phase

Before RAG can work, the documents need to be prepared:

  1. Chunking: Split documents into smaller sections (e.g., 500 tokens per chunk)
  2. Embedding: Convert each chunk into a vector
  3. Storage: Store vectors in a vector database (e.g., Pinecone, Weaviate, pgvector)

The quality of chunking has a major impact on results. Chunks that are too large dilute relevance; chunks that are too small lose context.

Step 4 of 4Reflect

Question · Self-check
Your company has 10,000 PDF documents with internal policies. Employees should be able to ask questions about them via chat. Which approach is most suitable?