What Changed After We Rebuilt Our Research Stack for Document Intelligence

On 2025-08-14 a production incident exposed a serious blind spot: our document intelligence pipeline-the system that ingests client PDFs, extracts tables and coordinates, and returns structured answers-fell into a performance plateau. The model-driven search layer that once matched requirements was now the limiter: long PDF chains caused noisy relevance signals, the literature-review step missed key citations, and engineering teams spent days chasing false positives instead of shipping features. As a senior solutions architect, the task was simple in brief and brutal in practice: recover accuracy, reduce time-to-insight for analysts, and make the research path repeatable under load. The Crisis: an operational plateau under load Our stakes were concrete. A fintech customer depended on our pipeline to extract compliance tables from multi-page PDFs; missed rows meant compliance reviews delayed, which directly increased manual audit time and escalated costs. The ingestion queue backed up d