When Your AI Wiki Outgrows the Context Window — A Practical Guide to RAG

Karpathy showed us how to build LLM-powered knowledge bases. But what happens when your wiki gets too big for the context window? Here's the missing piece. In a recent post , Andrej Karpathy described a workflow that resonated with thousands of developers: use LLMs to build and maintain personal knowledge bases as markdown wikis. Raw documents go in, the LLM compiles them into structured articles, and you query the wiki like a research assistant. He also noted something important: "I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries... at this ~small scale." The key phrase is "at this small scale." His wiki is ~100 articles and ~400K words. That fits in a large context window. But what happens when you hit 500 articles? 1,000? 2 million words? The context window runs out. Your LLM can't read everything anymore. This is where RAG comes in — and it's simpler than you think. What is RAG? RAG (Retrieval Augmented G