Step 1 of 4Knowledge
RAG is not a static concept. Since its introduction by Lewis et al. (2020), the architecture has evolved fundamentally. What started as simple "search and generate" is now a spectrum of architectures with different trade-offs.
Three generations have emerged:
- Naive RAG (2020--2023): Simple vector search + LLM generation
- Advanced RAG (2023--2025): Hybrid Search, Re-Ranking, Query Transformation
- Agentic RAG (2025+): LLM-driven, iterative retrieval loops
- Easy to implement
- Low latency
- Minimal infrastructure required
- No relevance optimization
- Keyword gaps with pure vector search
- Quality depends heavily on embedding model
- Irrelevant chunks in Top-K results
- Hallucinations from poor context
- Synonyms and paraphrases are missed
- Prototypes, simple FAQ bots, small document collections
Step 2 of 4Understand
Naive RAG -- The First Generation
Naive RAG follows a rigid three-step process: embed the query, vector search with Top-K, generate the answer. No query preprocessing, no re-ranking of results, no feedback loop.
Why it works: For simple questions with a single document as the answer, this is perfectly sufficient. "What is the return policy?" finds the matching paragraph and the LLM generates a correct answer.
Why it fails:
- Vague queries: "Tell me about returns" retrieves too many, overly unspecific chunks
- Multi-hop questions: "Which customers had both returns and complaints?" requires information from multiple documents that must be linked together
- Keyword mismatch: Semantic search finds "refund" for the query "money back", but misses exact terms like article numbers
Advanced RAG -- Hybrid Search and Re-Ranking
Advanced RAG addresses the weaknesses of Naive RAG through three key extensions:
1. Query Transformation Before the search starts, the query is optimized:
- Query Rewriting: The LLM rephrases the user's question to achieve better search results
- HyDE (Hypothetical Document Embeddings): The LLM generates a hypothetical answer, and its embedding is used for the search -- often closer to the target documents than the question itself
- Query Decomposition: Complex questions are broken down into sub-questions
2. Hybrid Search Combination of two search mechanisms:
- Dense Retrieval: Vector search via embeddings (semantic similarity)
- Sparse Retrieval: Keyword-based search like BM25 (exact matching)
- Fusion: Results from both searches are combined using Reciprocal Rank Fusion (RRF)
| Search Type | Strength | Weakness |
|---|---|---|
| Dense (Vector) | Semantic similarity, synonyms | Exact terms, codes, IDs |
| Sparse (BM25) | Exact matches, technical terms | Synonyms, paraphrases |
| Hybrid (Fusion) | Combines both | Higher effort, tuning required |
3. Re-Ranking After retrieval, a cross-encoder model re-evaluates each result in the context of the query. Cross-encoders are slower than bi-encoders (thus not suitable for initial search) but significantly more precise in relevance assessment.
Agentic RAG -- The Iterative Loop
Agentic RAG is the paradigm shift: the LLM actively controls the retrieval process. Instead of a rigid pipeline, the LLM decides:
- Whether to search at all (perhaps training knowledge is sufficient)
- What to search for (query formulation and reformulation)
- Where to search (which data source, which index)
- Whether the results are sufficient (or if another search is needed)
The loop works like a researcher working iteratively:
- Analyze the question and plan a search strategy
- Perform the first search
- Evaluate results: Are they sufficient? Are they relevant?
- If not: reformulate the query or consult a different source
- Repeat steps 2--4 until sufficient information is gathered
- Synthesize the collected information into a final answer
Agentic RAG is most powerful when the question is ambiguous or information from different sources needs to be synthesized. For simple lookup questions, it's overhead.
Step 3 of 4Apply
Decision Matrix: Which RAG Generation Do I Need?
| Criterion | Naive RAG | Advanced RAG | Agentic RAG |
|---|---|---|---|
| Question type | Simple lookup questions | Structured questions with technical terms | Complex, ambiguous questions |
| Data sources | One homogeneous source | One or few sources | Multiple heterogeneous sources |
| Latency budget | < 1s | 1--3s | 3--10s+ |
| Implementation effort | Low | Medium | High |
| Typical use case | FAQ bot, document search | Technical support, compliance | Research assistant, analysis |
Upgrade Path
In practice, you start with Naive RAG and upgrade strategically:
- Start with Naive RAG: Build MVP, measure baseline
- Measure retrieval quality: How often are the top-5 results relevant?
- Identify weaknesses: Keyword mismatches? Irrelevant chunks? Missing connections?
- Upgrade strategically: Hybrid Search for keyword problems, Re-Ranking for relevance, Agentic for multi-source
Step 4 of 4Reflect
You're designing a RAG system for a company with 50,000 technical documents, internal wikis, and ticket systems. Users ask both simple questions ("Where do I find form X?") and complex analysis questions ("Which incidents in the last 6 months involve product Y and have priority 1?"). Consider which architecture you'd choose and why. The quiz questions above help you sharpen your decision criteria.