Google Research has released the TurboQuant quantization compression algorithm, which can compress the KV cache of large language models to 3 bits, reducing memory usage by 6x and improving computational speed by 8x. The algorithm demonstrates excellent performance across multiple benchmark tests and aims to address model caching bottlenecks. It will be published at ICLR 2026.

Google Research has released the TurboQuant algorithm, which can compress the KV cache of large language models to 3 bits, reducing memory usage by at least 6 times while maintaining accuracy without requiring training. The algorithm optimizes traditional quantization through two sub-algorithms: PolarQuant and QJL. Testing shows excellent performance across multiple long-context benchmarks.