Cost-Aware Model Routing in Production: Why Every Request Shouldn't Hit Your Best Model

Your system isn't expensive because your models are expensive. It's expensive because every request defaults to the most capable model you have. That's not a cost problem. That's a routing problem. And most systems don't have a routing layer at all. Parts 1 and 2 of this series established why inference cost emerges from behavior, not provisioning, and why execution budgets are the enforcement mechanism that dashboards and alerts can never be. Part 3 is the decision layer that sits upstream of both: model routing. The control that determines which model handles each request — and why getting that wrong is the most expensive architectural default in production AI systems today. The Missing Layer Every inference request is an implicit classification problem: How much intelligence does this request actually require? Most architectures never answer that question. There is no decision layer between request and model. A request arrives. The model handles it. The model is always the same mode