PolarQuant: Quantizing KV Caches with Polar Transformation

A deep dive into how PolarQuant compresses LLM key caches by 4x using polar coordinates, and why it works so well. If you have ever tried running a large language model on long contexts (32K, 64K, or 128K tokens), you have hit the wall: the KV cache . It grows linearly with sequence length, eating up GPU memory and becoming the dominant bottleneck during inference. PolarQuant, introduced by researchers from KAIST, Google Research, and Yale ( arXiv:2502.02617 ), offers an elegant solution. Instead of quantizing key embeddings the usual way (in Cartesian space), it converts them to polar coordinates (angle and radius) and quantizes those instead. The result is a ~4x compression of the key cache with near-lossless quality on long-context benchmarks. Let's break down exactly how it works. What Is PolarQuant? Every time an LLM generates a token, it needs to attend to all previous tokens. To avoid recomputing everything from scratch, the model stores Key and Value embeddings in a cache. This