The context sizes of current models reflect the state as of November 2026. The strategies themselves are long-lived.
Step 1 of 4Knowledge
Every LLM has a context window -- the maximum amount of text it can "see" at once. Everything you send to the model (system prompt, chat history, documents, your current question) plus the response must fit within this window.
Context window sizes have evolved rapidly:
- 2022: GPT-3.5 had 4,096 tokens (~3,000 words)
- 2023: GPT-4 launched with 8K tokens (March 2023), GPT-4 Turbo introduced 128K tokens (November 2023)
- 2024: Gemini 1.5 Pro reached 1M tokens (~750,000 words)
- 2025/26: 1M tokens becomes standard for top models -- such as Claude Opus 5.5, Gemini 4 Argon, and GPT-6 (approx. 1M)
But a large context window alone doesn't solve all problems. More context means higher costs, longer response times, and the risk that important information gets "lost" in the middle (the so-called Lost in the Middle problem).
Step 2 of 4Understand
Strategy 1: Sliding Window
With the sliding window approach, the model only keeps the last N messages in context. Older messages are discarded. This is the simplest strategy and works well for short conversations.
Advantage: Cheap, fast, predictable. Disadvantage: The model "forgets" early parts of the conversation.
Strategy 2: Summarization
A smarter approach: instead of discarding old messages, an LLM summarizes them. The summary is kept as context while the original messages are removed.
Advantage: Important information is preserved. Disadvantage: Summaries can lose details; additional API calls required.
Strategy 3: RAG (Retrieval-Augmented Generation)
Instead of cramming everything into the context, RAG retrieves only the relevant information from an external database. The search query determines which documents are loaded into the context window.
Advantage: Scales to millions of documents; context stays focused. Disadvantage: Requires infrastructure (vector database, embedding pipeline).
Small contexts (under 10K tokens): Sliding window is enough. Medium contexts (10K-100K): Summarization helps. Large knowledge bases (100K+ documents): RAG is the right approach.
Step 3 of 4Apply
Imagine you're building a chatbot for customer support at an online store. The chatbot should:
- Have access to 500 FAQ articles
- Take the chat history with the customer into account
- Respond quickly and cost-effectively
The optimal solution combines strategies: RAG for the FAQ articles (load only relevant ones), sliding window for the current chat history, and optionally summarization for longer support sessions.