RAG Is a Data Problem Before It’s a Prompt Problem

I made this mistake myself while debugging a RAG pipeline. If your RAG feature keeps returning plausible but wrong answers, inspect retrieval before you touch the prompt again. I learned that only after spending time on the wrong lever. I rewrote the prompt several times, added constraints, tightened the wording, and told the model to stay closer to the supplied context. The answers sounded better. They were still wrong. The fix was not a smarter prompt. The fix was cleaning the data path: removing stale documents, changing chunk boundaries, adding usable metadata, and checking what retrieval actually returned. This post is based on that debugging experience, not a benchmark study. My claim is narrower than “prompts do not matter.” They do. But in the kind of production RAG systems many of us build, retrieval failures often show up as answer quality failures, so they get misdiagnosed as prompt problems. The Failure That Looked Like a Prompt Bug The setup looked reasonable on paper. I h