Nvidia's DMS shrinks LLM memory up to 8x without hurting reasoning
Nvidia researchers just introduced Dynamic Memory Sparsification (DMS), a technique that compresses LLM KV cache by up to 8x without degrading accuracy. Instead of rigid deletion heuristics, DMS trains models like Llama 3 or Qwen 3 to intelligently decide which tokens to drop using a delayed-eviction window. The feature is available in Nvidia's Model Optimizer framework.