Building a Production RAG Pipeline: Architecture Decisions That Matter

Most RAG tutorials show you a happy path. Chunk text, embed it, retrieve the top-k, stuff it into a prompt, ship it. That gets you 80% of the way there. The other 20% is where your system breaks in production: wrong embedding model, bloated context windows, synthesis models hallucinating outside retrieved facts, no fallback when Bedrock returns a 500. I built Scout, an AI company research agent, for the Amazon Nova AI Hackathon. The core of it is a RAG pipeline that takes data extracted from five websites, embeds the resulting briefings, and enables semantic search across a history of past research. This post covers the architectural decisions that actually mattered. The Setup Scout has two AI tasks that look similar but have completely different requirements: Take raw scraped data from five sources (company website, LinkedIn, Crunchbase, Google News, job listings) and synthesize it into a structured briefing Store each completed briefing and let users search across them semantically (