AI Home Lab — Part 3: Building a RAG Pipeline: Making Your Local AI Actually Know Your Stuff

In Parts 1 and 2, we set up Ollama with phi3:mini and wired up Prometheus and Grafana to monitor it. The model was running, but it only knew what it was trained on. In this part, we fix that — by building a RAG pipeline that lets the model answer questions about our own docs, configs, and playbooks. What is RAG and Why Does It Matter? If you've ever asked a local LLM about your own infrastructure and got a generic answer, you've hit the core limitation — the model simply doesn't know about your setup. It was trained on public data, not your Ansible playbooks or your Prometheus configs. RAG stands for Retrieval-Augmented Generation . The name sounds complex but the idea is simple: Instead of expecting the model to have memorised everything, you hand it the relevant information right before it answers. Think of it like an open book exam. The model doesn't learn anything new — it just gets to read the right page before writing its answer. RAG solves two problems: The model's knowledge has