Why Routing Isn't Governance: The Missing Layer in AI Agent Infrastructure

The AI gateway market is booming. Bifrost, LiteLLM, Portkey, and others are racing to solve the same problem: how do you efficiently route LLM calls across multiple providers? It's a real problem. When you're building AI applications, you don't want to be locked into a single provider. You want failover when OpenAI goes down. You want load balancing across API keys. You want semantic caching to reduce costs. These gateways solve that beautifully. They're fast, reliable, and well-engineered. But they solve the wrong problem for the agent economy. The Routing Mindset Routing gateways think about the world in terms of requests: Which provider should handle this request? How do we minimize latency? How do we maximize uptime? How do we cache similar requests? These are infrastructure questions. They're about reliability and performance. And they assume a human is ultimately in control — making decisions about which applications to build, which APIs to call, and how much to spend. The Agent