
Benthos Has a Free Stream Processor — Connect, Transform, and Route Data Between Services
Benthos (now Redpanda Connect) is a declarative stream processing tool for ETL, routing, and transformation. What You Get for Free 200+ connectors — Kafka, RabbitMQ, S3, PostgreSQL, HTTP, NATS, and more Processing pipeline — filter, map, parse, enrich data in-flight Bloblang — powerful data transformation language Error handling — dead letter queues, retry strategies Batching — configurable batch sizes and timing Rate limiting — control throughput per pipeline Metrics — Prometheus metrics out of the box YAML config — entire pipeline in one config file Single binary — no dependencies Quick Start # pipeline.yaml input : kafka : { addresses : [ " localhost:9092" ], topics : [ " raw-events" ] } pipeline : processors : - bloblang : | root.user = this.user_id root.action = this.event_type.lowercase() root.timestamp = now() output : postgresql : { dsn : " postgres://user:pass@host/db" , table : " events" } benthos -c pipeline.yaml Why Data Engineers Choose It Custom ETL scripts are fragile. K
Continue reading on Dev.to DevOps
Opens in a new tab




