Skip to content
IFAI

TurboQuant

Product2 stories

  1. Open sourceReach 48

    Open-source RotorQuant beats Google at KV cache compression

    A new open-source algorithm called RotorQuant just outperformed Google's TurboQuant across every metric. By swapping heavy global transforms for compact Clifford rotators, it hits 10x KV cache compression while boosting prefill speed by 5.3x and decoding by 28%. Why it matters: massive 128K context windows that previously required multiple enterprise GPUs can now fit onto a single 24GB card.

  2. ResearchReach 36

    Google squeezes LLM memory with TurboQuant

    Google Research just unveiled TurboQuant, a new quantization algorithm that shrinks key-value caches down to 3 bits with zero accuracy loss. By ditching the traditional memory overhead of quantization constants, it cuts KV memory by at least 6x and runs up to 8x faster on H100 GPUs. Why it matters: running massive long-context models just got substantially cheaper and faster.