The API Was “Fast” — Until One User Made It Slow for Everyone

We had an API that was consistently fast. ~80ms average response time. No complaints. Then one day, everything slowed down. Not crashed. Not broken. Just… slow. And the weird part? It was caused by one user . The Symptom: Latency Gradually Creeping Up At first: p95 latency increased slightly CPU was fine Memory was stable No obvious errors Then: Requests started taking 500ms+ Some hit 2–3 seconds But only during certain times It wasn’t global traffic. It was something more subtle. The Clue: One Endpoint, One Pattern After digging into logs, we noticed: Almost all slow requests hit the same endpoint Same query pattern Same user ID appearing frequently That user had… a lot of data. Way more than anyone else. The Root Cause: “Works Fine” Query That Didn’t Scale Here’s the query (simplified): SQL: SELECT * FROM orders WHERE user_id = $ 1 ORDER BY created_at DESC ; Looks harmless, right? Except: No pagination No limit That user had 120,000+ rows Every request: Pulled all rows Sorted them Se