Alibaba drops Qwen3-Next-80B-A3B: 80B scale with 3B compute

Massive capacity, lightweight footprint

Running frontier models usually means paying frontier-sized GPU bills. Alibaba's Qwen team just dropped Qwen3-Next-80B-A3B, an architecture designed to break that equation.

The model packs 80 billion total parameters, but activates just 3 billion per token. Alibaba claims this cuts training costs by 10x and boosts inference speeds by 10x compared to Qwen3-32B, with gains especially noticeable on contexts over 32K tokens.

Under the hood, it pairs a hybrid Gated DeltaNet and Gated Attention design with an ultra-sparse MoE setup featuring 512 experts, routing just 10 routed experts and one shared expert per step. It also adds Multi-Token Prediction to accelerate speculative decoding. Benchmark-wise, Alibaba says the Thinking variant outpaces Gemini-2.5-Flash-Thinking, while the Instruct version approaches their 235B flagship.

Why it matters

We think this marks a critical milestone for ultra-sparse architectures. Activating just 3B parameters means developers can tap into 80B-class reasoning without needing massive multi-GPU clusters for serving.

It puts pressure on mid-sized dense models by proving you don't have to choose between long-context recall and lightning-fast throughput.

That said, managing routing across 512 experts is notoriously tricky for memory bandwidth. We suspect real-world teams will need to carefully test their specific serving setups to see if they can truly achieve the claimed 10x speedups outside Alibaba's optimized environment.