Monitoring Self-Hosted LLM with Prometheus and Grafana

Audience: Intermediate DevOps | Series: Part 2 of 4 Quick Recap from Part 1 Set up Ubuntu Server VM (phi) on VirtualBox Installed and configured Ollama as a systemd service Automated entire setup with Ansible (llm-ansible repo) Interacted with phi3:mini via CLI, curl Link to Part 1 Why custom monitoring setup Ollama does not have a native Prometheus exporter (a /metrics endpoint) primarily because it is designed as a lightweight, user-friendly tool for running local LLMs, focusing on simplicity and ease of setup for local developers rather than complex enterprise monitoring What This Post Covers Writing a custom Prometheus exporter in Python Installing Prometheus and Grafana with Ansible Building a monitoring dashboard for your LLM Github Link Repository Link Section 1 — The Problem 1.1 Ollama Has No Native Metrics Most production services expose a /metrics endpoint in Prometheus format out of the box. Ollama does not. curl http://192.168.1.52:11434/metrics # 404 page not found This is