Understanding Vector Pipelines: From Config Files to Data Flow

Part 2 of a series on building a metrics pipeline into ClickHouse (Read Part 1: Why my metrics pipeline with Telegraf didn’t work) Picking Up Where Things Broke In the previous part, I talked about trying to build a metrics pipeline using Telegraf - and why that approach didn’t work for my use case. The biggest issue wasn’t just tooling. It was this: I didn’t have enough control over how data moved through the system. That’s what led me to explore a different approach. Why Vector I came across Vector while looking for something more flexible. At a glance, it felt different. Instead of thinking in terms of plugins and configs, Vector is built around a pipeline model . And that changes everything. The Core Idea: Pipelines At the center of Vector is a simple concept: Sources → Transforms → Sinks That’s it. But this model makes the flow of data explicit. Sources → where data comes from Transforms → how data is modified Sinks → where data is sent Compared to my earlier approach, this immedi