Built a Caching Proxy for OpenAI — Saved 40% on API Bills

A maintenance manager's first SaaS. Technical deep-dive + lessons learned. Hey dev.to! 👋 I'm not a career developer. I supervise industrial mechanics and run a maintenance department. But we needed AI for our CMMS (Computerized Maintenance Management System), and the OpenAI API costs were getting crazy. So I built a caching proxy. Here's how it works, what I learned, and the actual code. ─── The Problem We're using AI for: • Auto-generating work orders • Predictive maintenance alerts • Vendor communications • Training docs Issue: Same prompts, repeated constantly, paying every time. User: "Generate work order for HVAC maintenance" → Pay $0.002 User: "Generate work order for HVAC maintenance" (same prompt) → Pay $0.002 again User: "Generate work order for HVAC maintenance" (same prompt, 3rd time) → Pay $0.002 AGAIN This adds up FAST at scale. ─── The Solution: Caching Proxy Intercept OpenAI requests, hash the prompt, cache the response. Architecture: Your App → AI Optimizer Proxy → Open