Designing a Tool Architecture for AI Agents — Base Tools, Toolkits, and Dynamic Routing

How do you give an AI agent 30+ tools without drowning the context window? Include everything and you waste tokens. Be selective and the agent can't do its job. Here's how I solved it with a 3-layer architecture. The Problem: Too Many Tools As your AI agent grows, so does its toolbox. My personal assistant now has 35+ tools — web search, email, calendar, weather, Git, host PC control, file management, code execution, and more. Sending all 35 tool schemas to the LLM on every request causes two problems: Token cost explosion : 35 JSON function schemas easily consume 3,000+ tokens per turn Selection accuracy drops : The more tools available, the more likely the LLM picks the wrong one But if you trim tools aggressively, the agent can't handle requests it should be able to. The Solution: 3-Layer Architecture ┌───────────────┐ │ User Input │ └───────┬───────┘ ↓ ┌────────────────────────────────────┐ │ Tool Registry │ │ │ │ ┌────────────┐ ┌──────────────┐ │ │ │ Base Tools │ │ Toolkits │ │ │