Saving $300/Month While Fixing WebRTC Drops: How Istio's Consistent Hashing Beat Redis

I Spent hours Debugging Sessions Until I Discovered Istio Consistent Hashing A developer's journey from late-night debugging to elegant infrastructure solutions The Problem That Started a Journey We had a scaling problem that wouldn't go away. Our Jarvis service needed to grow. We were scaling from 3 pods to handle more AI inference requests and audio interactions. But there was a critical issue: each Jarvis pod maintained state—model weights loaded in memory, WebRTC connections via daily.co for live audio calls, cached inference results. When requests bounced randomly between pods (thanks, round-robin load balancer), things broke: Request 1 (user-123) → Jarvis-Pod-1 ✓ Model weights loaded, WebRTC connection active Request 2 (user-123) → Jarvis-Pod-2 (different pod!) ✗ "Connection lost" - WebRTC drops ✗ Model weights reloaded - 500ms latency spike ✗ User's audio call freezes mid-session Request 3 (user-123) → Jarvis-Pod-1 ✓ Works again, but connection restarted Month 1: We implemented