Google's Cache Compression, Siri's Open Door, and the CFO Agent Test

Google's Cache Compression, Siri's Open Door, and the CFO Agent Test Google slashes AI memory needs by 6x while Apple opens Siri to rivals, and a new AI-focused language arrives as LLM agents face their first CFO benchmark. Google's TurboQuant reduces AI LLM cache memory capacity requirements by at least six times — up to 8x performance boost on Nvidia H100 GPUs, compresses KV caches to 3 bits with no accuracy loss - Tom What happened: Google's TurboQuant cuts LLM cache memory requirements by at least six times, achieving up to 8x performance gains on Nvidia H100 GPUs by compressing KV caches to just 3 bits without accuracy loss. Why it matters: This breakthrough directly addresses the memory bottleneck that limits LLM inference speed and scale, potentially enabling larger models to run on existing hardware or dramatically reducing infrastructure costs for AI services. Context: KV cache compression has been a key focus area as models grow larger and inference costs become prohibitive f