Why My Metrics Pipeline with Telegraf Didn’t Work (and What I Learned)

Part 1 of a series on building a metrics pipeline into ClickHouse Collecting metrics is easy. Shipping them to an analytical database without losing your mind is the hard part. The Goal At one point, the task seemed straightforward: Collect system metrics (CPU, memory, GPU) and store them in ClickHouse for analysis. This is a common observability use case. You collect metrics, send them somewhere, and run queries on top. Simple enough. But in practice, it didn’t go as planned. The Initial Approach: Telegraf I started with Telegraf. It’s widely used for collecting system metrics and has a plugin-based architecture, which makes it a natural first choice. This was also where I first came across TOML. At first, it felt like I just needed to “write a config and run it.” But very quickly, I realized: Configuration isn’t just syntax-it defines how your system behaves. What I Was Trying to Build The idea was simple: Collect host-level metrics (CPU, memory, etc.) Collect GPU metrics Push everyt