Line of Defense: Three Systems, Not One

Three Systems, Not One "Rate limiting" gets used as a catch-all for anything that rejects or slows down requests. But there are actually three distinct mechanisms, each protecting against a different failure mode, each asking a different question: Mechanism Question it asks What it protects Load shedding "Is this server healthy enough to handle ANY request?" The server from itself Rate limiting "Is THIS CALLER sending too many requests?" The system from abusive callers Adaptive throttling "Is the DOWNSTREAM struggling right now?" Downstream services from this server A rate limiter won't save you when your server is OOM-ing — every user is within their quota, the server is just dying. Load shedding won't stop one customer from consuming 80% of your capacity — total concurrency is fine, the distribution is unfair. And neither will prevent you from hammering a downstream service that's already struggling. These are complementary systems. Treating them as one thing — or building only one o