Run Your Own Local AI Chat with OpenWebUI and llama.cpp - Windows

TL;DR: A local ChatGPT-like stack using OpenWebUI as the UI and llama.cpp as the inference server, with a GGUF model from Hugging Face. Everything talks over an OpenAI-compatible API. No API bills, no data leaving your machine. Why this matters Privacy: Prompts and replies stay on your machine. No API bills: No usage-based pricing or quotas. Control: You pick the model, quantization, and context size. Open source: OpenWebUI and llama.cpp are free and auditable. I wanted a local tool for LLM tasks that don't need a paid API: drafts, small scripts, experiments. This setup does that. Who this is for Anyone who wants a local AI chat without subscriptions. No prior LLM experience required; this is mostly wiring a UI to a local server. My setup (Windows) OS: Windows 11 RAM: 16 GB minimum; 32 GB helps for larger models GPU: optional but recommended for speed (I have a GPU with 8 GB of VRAM) Disk: Enough for multi-GB model files (often 4–8 GB per model) A 7B model in Q4 quantization runs on ma