ISTA-DASLab just released non-uniform GSQ-RCO GGUF quantizations for DeepSeek-V4.1-Flash-1017, compressing the 763B MoE down to as low as 1.75 bpw. Paired with the Strata inference engine, the massive model runs on a single consumer 12GB GPU while the 2.00 bpw build retains 100% of the base model's task performance. Why it matters: Running frontier-tier open models locally no longer requires a server rack.
A new open-source algorithm called RotorQuant just outperformed Google's TurboQuant across every metric. By swapping heavy global transforms for compact Clifford rotators, it hits 10x KV cache compression while boosting prefill speed by 5.3x and decoding by 28%. Why it matters: massive 128K context windows that previously required multiple enterprise GPUs can now fit onto a single 24GB card.
Local LLMs just became real on everyday hardware. Thanks to Georgi Gerganov’s new llama.cpp and 4-bit quantization, Meta’s LLaMA can now run directly on personal laptops like a MacBook. It brings GPT-3-class capabilities straight to individual developers, breaking AI out of the corporate cloud garden.
