Celery Is Overkill: When to Use Redis Pub/Sub Instead

Originally published at https://prodevs.in/tutorials/celery-is-overkill-when-to-use-redis-pub-sub-instead/ . A client’s order‑processing pipeline suddenly hit 12 k jobs/min, causing Celery workers to stall, spill un‑acked tasks, and eventually crash. The root cause wasn’t a Celery bug; it was the overhead of a full‑blown distributed task system on a tiny VPS. By swapping the Celery broker and workers for a simple Redis Pub/Sub channel, the team reduced memory usage by ~300 MiB and eliminated the crash. The fix boiled down to four lines of code, but the debugging marathon lasted six hours because the team chased Celery‑specific metrics instead of looking at the real bottleneck. 4 Key Takeaways Celery isn’t always the default choice – it brings a broker, result backend, worker pool, and beat scheduler, each adding latency and operational cost. Fire‑and‑forget, low‑latency workloads often don’t need retries, routing, or persistence – a lightweight Pub/Sub layer suffices. Resource‑constrai