How I Built an AI-Powered Error Triage System for SaaS at Scale — And What It Actually Costs

We had a monitoring problem that wasn't really a monitoring problem. We had Datadog. We had alerts. We had dashboards. What we didn't have was signal. On any given morning, an engineer opening the console might see a large volume of errors aggregated across many customer environments — with no fast way to know if that was one cascading timeout firing repeatedly, or a dozen distinct failures quietly spreading across the fleet. I built an internal production dashboard to surface that signal. Then I added AI-powered error analysis to it. The pipeline runs on a schedule throughout the day. Here's the architecture, the reasoning, and illustrative code for each layer — patterns you can adapt; they are not copy-pasted from a private repo — including the part many AI monitoring write-ups skip: who owns the problem once the AI summarizes it. The Problem With Raw Error Counts The product is SaaS, but it is not the classic “everyone on one shared multi-tenant stack” shape: customers run in separa