How to design a Real-Time Chat Application

Why Designing a Real-Time Chat Application Is Hard Designing a real-time chat application is significantly more complex than building systems like a URL shortener or a notification service. The main reasons are: Real-time bidirectional communication Handling millions of concurrent connections Ensuring low latency Managing message persistence and offline delivery Unlike simple request-response systems, chat applications require persistent connections and instant delivery at scale. Functional Requirements 1-to-1 messaging Group messaging Message persistence Offline message delivery (messages should be delivered when a user comes online) Non-Functional Requirements Scalable to millions of users Low latency (< 500 ms) Fault tolerant Highly available Durable storage Choosing the Correct Communication Protocol Since our latency requirement is less than 500 ms, traditional short polling or long polling are not ideal because they introduce unnecessary delays and overhead. Server-Sent Events (S