The Gateway Pattern - One API, Any Model

After our Lambda approach fell apart, I needed a new architecture. Something that could handle any AI provider through one clean API. Something that could stream properly. Something that wouldn't fight us at every turn. The solution was the gateway pattern. One API endpoint that normalizes requests across all providers. Send the same JSON payload whether you're using OpenAI, Claude, or Bedrock. The gateway handles provider-specific formatting, retries, fallbacks, and streaming. Why Not Just Use LiteLLM? Before building our own, I seriously considered LiteLLM. It's a clever proxy that does exactly this - normalize API calls across providers. But we had specific needs: TypeScript-first : Our frontend team lives in TypeScript. I needed strong typing across the entire stack. AWS CDK deployment : Everything deploys through CDK. I needed infrastructure as code. Cost tracking : Built-in tracking per request, per model, per application. LiteLLM doesn't handle this. Custom auth : Integration wi