Open-source RotorQuant beats Google at KV cache compression
A new open-source algorithm called RotorQuant just outperformed Google's TurboQuant across every metric. By swapping heavy global transforms for compact Clifford rotators, it hits 10x KV cache compression while boosting prefill speed by 5.3x and decoding by 28%. Why it matters: massive 128K context windows that previously required multiple enterprise GPUs can now fit onto a single 24GB card.