How I built an OpenAI-compatible API layer on top of Ollama (and what broke along the way)

I've been building NestAI for the past few months — a platform that deploys private Ollama + Open WebUI servers for teams in about 33 minutes. Recently shipped an OpenAI-compatible API layer on top of it and wanted to share what the journey looked like, including the parts that broke silently at 2am. Why OpenAI-compatible The obvious reason: adoption. Most developers already have OpenAI code. LangChain integrations, existing chatbots, internal tools. If switching to a private AI stack means rewriting everything, most teams won't bother. So we made it a one-line change: pythonfrom openai import OpenAI Before client = OpenAI(api_key="sk-...") After — everything else stays identical client = OpenAI( base_url=" https://nestai.chirai.dev/api/v1 ", api_key="YOUR_NESTAI_KEY" ) Same SDK. Same methods. Same response format. Just your own infrastructure. The stack Each NestAI server is a dedicated Hetzner Cloud VM running: Ollama — local model inference Open WebUI — chat interface + API layer ng