Monitoring an ML Pipeline in Production: Anatomy of an Open-Source Stack

This isn't a theoretical guide. It's a field report on the observability stack I've built and iterated across industrial engagements and demos on the AI Observability Hub - a demonstration platform I use to validate AI monitoring architectures before deploying them at client sites. The goal is straightforward: give an SRE, data engineer, or CTO the building blocks to monitor an ML pipeline in production with VictoriaMetrics, OpenTelemetry, and Grafana. No vendor lock-in. No proprietary platform. Open-source components, assembled with intention. What we actually monitor (and what we forget) Most organizations deploying ML in production settle for monitoring infrastructure: CPU, RAM, disk space. That's necessary, but it's the equivalent of watching a factory's temperature without looking at the quality of parts coming off the line. A production ML pipeline has four observability layers : Infrastructure : the foundation. GPU utilization (compute, VRAM, memory bandwidth), CPU, network, dis